So the Haystack Affair (is there a Wikipedia page named after this already?) continues generating food for thought for those of us working at the intersection of free expression, Internet censorship, and media development.

Yesterday I blogged about what the Haystack Affair suggested about the responsibility of "Internet intellectuals." Ethan Zuckerman, who was one of the intellectuals I singled out in that post, eloquently responded to my criticism on his blog.

"I’ve not published on Haystack for a very simple reason: I haven’t been able to conduct a proper evaluation of either the tool or the protocols behind it," wrote Ethan.

But I think that Ethan's rebuttal fundamentally misunderstood the origins and the direction of my original criticisms. Ethan writes:

"Evgeny’s concern in his recent post appears to be that I haven’t publicly critiqued Haystack, a proposed censorship circumvention tool that’s received a great deal of laudatory press coverage."

That's not my concern; I apologize if I didn't state it more cogently in the original post. My concern is that Ethan hasn't joined the public pressure campaign on Haystack to a) open up their code to external examination b) provide more information about how they operate.

Once again, I know that this post will make me sound extremely self-righteous so there is no need to point that out in the comments (I'm looking at you, David Sasaki!).

To understand what exactly I'm driving at, I need to elaborate on the history of the Haystack Affair. Here is my brief understanding of what has happened. (I'm almost sure I am overstating my own role in this -- but I hope Jake Appelbaum and others can chime in the comments and set the record straight.

***

Around early August 2010, many people on Stanford's Liberation Technology mailing list started asking a lot of serious questions about Haystack's model; their unwillingness to have their code examined; and their overall lack of transparency.

Based on my own long-running investigation of Haystack -- which long predates my membership in the Lib Tech mailing list (I discuss Haystack in my book, so I did a lot of research around them in the past) -- I wrote a provocative blog post, which got some media attention and triggered an angry -- some might say nearly hysterical -- reaction from Austin Heap.

This, in turn, led to even more questions about Haystack on the Lib Tech mailing list and elsewhere in the blogosphere. Austin's nervous reaction surprised me, prompting me to continue digging into Haystack. It also pushed me to join forces with Jake Appelbaum, who has had many similar concerns, and voiced support to my criticism of Haystack on the Lib Tech mailing list.

I also started working on a second, much longer blog post for Foreign Policy. I decided to indulge Austin's call for more and better reporting around Haystack and put my heart into the investigation.

In the meantime, the attention that all of this brought to Haystack pushed Austin to release some major information about its inner workings in responses to my questions. For the first time, we actually got a short technical note from Haystack's developer Daniel; it discussed some basics about Haystack's steganography.

I believe such close scrutiny of Haystack -- in my blog posts and by others in the blogosphere and on the mailing list -- has also pushed Austin Heap to demo the software to more people in San Francisco. Some of them noticed flaws in Haystack's design. (Of course, it's also possible that Austin had been planning those meetings for a long time; the fact remains that the fact that Haystack was everyone's favorite subject last week probably resulted in much more scrutiny being paid to how well it matches the claims made by its founders. )

Jake gleaned some information about Haystack from my communication with Heap (much of it was on the record -- and Austin himself promised to publish his answers) and conducted his own investigation. We did share a lot of notes in the process -- it was a collaboration I greatly enjoyed! In addition, some of the details that emerged from the demos that Austin Heap showed to people in San Francisco helped Jake grasp that it was extremely insecure and push Austin to shut it down (which, as we later found out, he didn't).

I kept digging into Haystack, reading everything I could find and spending a lot of time talking to people on Skype and on the phone. Eventually I got hold of several Iranians in London who had helped Austin Heap to recruit some of Haystack's developers in the country. It was them who gave me a copy of Haystack, which I then passed on to Jake for analysis. You all know the rest: Haystack was shut down on Sunday night.

Now, my conclusion -- and I'm curious as to what Jake thinks about this - is that we wouldn't have been able to expose Haystack without the public campaign that aimed at forcing Austin to start sharing more information about the project.

Even before seeing a copy of their code, Jake knew enough about how Haystack works to force Haystack to shut down. What he learned on Sunday was devastating -- but we knew ENOUGH by end of day Friday -- BEFORE we got hold of a copy of Haystack's code.

Thus, where I do think Ethan and many other academics/intellectuals failed to act responsibly was in not joining that public pressure campaign on Austin -- on the mailing list or on their blogs and tweets (not to be a complete jerk about it but Ethan's only public noise about Haystack was a retweet of someone else's suggestion that Austin Heap and me go on Jerry Springer).

Anyone who would go through that Lib Tech mailing list would not fail to notice that many of the questions raised about Haystack -- once again without anyone seeing the code -- were valid. I don't think it was a particularly hard campaign to notice -- and I think that Ethan himself acknowledges that he has been watching it from the sidelines.

Could Austin Heap be pressured into opening up his virtual (in all senses) empire earlier had people like Ethan Zuckerman joined our efforts? I don't know -- but I do think that Ethan and others did HAVE a responsibility to join that debate and voice their concerns about Haystack's methods and strategy.

One doesn't have to look inside Haystack's code to notice that the kind of risks its founders were putting their users under required a completely different operating model and probably a different working relationship with the rest of the community.

Evaluation of technology requires more than just close scrutiny of the code and the protocols involved; it also requires some hard thinking about the appropriate norms & the context. Academics -- and especially academics with a public profile who take it upon themselves to explain technology to non-technologists -- are well-aware of what those norms and contexts are. As far as I am concerned, it seemed pretty clear that Haystack violated both. If Ethan and others can make a convincing case that this was NOT clear, I'll be happy to acknowledge that I'm wrong and retract my criticism.

On the other hand, let's just imagine what would have happened if that public campaign had NOT occurred. Chances are that Austin would still be meeting with senators, raising money, and putting even more Iranians at risk. Ethan and others would still be waiting until a copy of Haystack's code would suddently drop from the sky.

To know that people's lives may have been put at risk and fail to act when such an opportunity to came around -- well, I just don't think that this is a valid excuse. I don't want to go all Sartre on Ethan or anyone else at the Berkman Center and outside, but to me it seems quite obvious that the noble desire to publish respectable papers about how circumvention technology works does not absolve one of the necessity to engage in public debate about it, especially when one has so much to contribute to it. One doesn't have to stop being an academic to participate in such debates; I don't think that a couple of critical emails to a mailing list somehow compromise anyone's academic integrity.

(And I do find it quite hilarious that two people without a PhD -- me and Ethan -- are arguing about the responsibility of academics. For the record, I deliberately framed this as the “responsibility of intellectuals” debate for that purpose.)