Hacked By Kosova Elite Hackers

I was doing some experiments with Adam in the lab on Friday, and we discovered some interesting variations in the way that Firefox and Safari implement the HTML 5 Cache Manifest specification. I think this is a particularly important feature to have implemented consistently across platforms because it is the make-or-break feature of HTML5 that will permit web applications to function offline.

First, what is the manifest?

For people who haven’t heard about this feature before, the manifest is essentially a special file that lists portions of a web site that should be cached locally for offline access. This is the feature of HTML 5 that will standardize the type of “airplane mode” access that GMail users have with Google’s custom Gears plugin.

The manifest is served as a regular old file, with MIME type text/cache-manifest, and is linked from the html tag itself, as follows:


<html manifest="site.manifest">

..

</html>

Once a web site is marked as being cached, then the browser will use the local cached copy of all the files specified in the manifest instead of attempting to load them from the internet. Say you’re on an airplane and type in the URL for http://my_cached_site.com. The browser will recognize it as a cached one, load it from its local storage instead, and then use a new JavaScript API to inform the web site that it is running in offline mode.

So now for the important part, how do these two browsers (Firefox and Safari) handle this file?

Firefox

Upon loading an HTML5 document with a manifest attached, Firefox firsts asks permission to cache the site offline before requesting the manifest file from the server. Here is how the toolbar looks on my browser:

cache_ffox_permission

And here is the server log (I’m using a Rails project to test this) to show that the manifest was not yet loaded:

cache_firefox_first_load

If you choose to allow offline caching, the web browser then requests the cache file, as can be seen from this screen shot.

cache_ffox_after_perm

Now here’s the cool thing, I set the headers on the manifest file such that the manifest file itself should also be cached on the client side:


        headers["Expires"] = "Fri, 30 Oct 2010 14:19:41 GMT"
        headers["Cache-Control"] = "max-age=3600, must-revalidate"

And the result of this is that the subsequent load, no files at all are loaded from Firefox — it operates entirely offline. Notice the completely empty server log as I reload the site 2..n times.

cache_ffox_second_load

Safari

Now let’s look at how Safari does it. Upon loading the web page, Safari also does not load the manifest file, as can be seen from this screen shot:

cache_safari_first_load

However, it also does not ask any questions about offline access. The next time I load the web page, something strange happens. Safari checks the manifest file twice and then doesn’t load the actual HTML page (because it doesn’t have to). The double-loading of the manifest file appears to be on the second page load, not split 1/1 between the page departure and subsequent reload. A little strange, if you ask me.

cache_safari_second_load

Furthermore, when I reload the page, despite the HTTP headers specifying that the manifest should be cached, Safari reloads the manifest file. Though, at least it only loads it once for every subsequent time:

cache_safari_third_load

Conclusion

I’m no spec-master, but it seems like Firefox’s implementation of this feature is what I would want to happen as a web architect, while Safari’s behavior seems a bit strange.

Firefox:

Only loads the web page once
Asks the user for permission to enter offline mode
Only downloads the manifest file once if given permission
Then obeys HTTP Cache Control headers to suppress reloading the manifest file on future loads

If Safari were to also behave like this, there are a few fixes that need to be implemented. Namely:

Ask the user if offline access should be allowed
Load the manifest when the user loads the page the first time (and approves offline mode), not the second time, when the user might be on an airplane
Stop loading the manifest file multiple times in a single page load
Start obeying the HTTP cache headers so that zero web connections are necessary if the cache says so

Safari’s Manifest handling quirks aside, both browser teams should be applauded for so aggressively implementing the HTML5 spec. It is a real treat as someone researching web platforms to get to test the in-progress spec on real browsers instead of just talking about what might eventually happen down the road.

This article is cross-posted on the Haystack Blog.

Written by Ted

September 26th, 2009 at 2:39 pm

Posted in The Interwebs, Web Development

Tagged with HTML5

Will the Namespace Traffic Jam Kill RDFa in HTML5?

without comments

[ Cross-posted on the Haystack Blog ]

One of the most exciting aspects of the (in-progress) HTML5 specification is the number of data-centric features it contains. It’s almost as if the committee is saying a big, “OK, OK! We heard you!” to all the data-heads out there and is providing not one, not two, not three, but four different ways to access and manage structured data from within the client browser:

Data Attributes, are key-value pairs that may be added to any DOM node
Microdata provides a way to interweave objects and object-properties amidst the DOM
RDFa provides a way to interweave RDF amidst the DOM
Client-side Database Support provides a full relational data access from JavaScript (the spec says this will be SQL compliant, but in reality it will likely just be the SQLite subset of SQL).

These are all great developments, and will no doubt bring about a lot of creativity about how data can be used on the client-side, but what interests me the most is why the HTML5 working group felt the need to include Microdata alongside RDFa.

The capabilities of HTML5 Microdata and RDFa are nearly identical, albeit with slightly different terminology. Both provide a way to embed data within HTML attributes and tag contents. Both allow for both named entities and blank nodes. And both allow for a variety of more complex constructions, such as lists and HREF property values. One of the only real differences, as I can tell from glancing over the specs, is that RDFa requires URIs whereas Microdata simply uses ordinary strings to reference entities and properties. And that is what worries me: one of the biggest benefits of RDF is its use of URIs, yet URIs seem to be exactly what is preventing the adoption of RDF.

One problem is probably that URIs look funny as data model elements, even to a programmer. “A person has name” is much more natural sounding than “A http://csail.mit.edu/Contact#Person has a http://csail.mit.edu/Contact#name”. We think of our code in natural language terms, and URIs obfuscate our real world metaphors.

Far more serious a problem is the namespace traffic jam that currently exists. If I want to publish an RDF document that describes this blog, for example, best practice would have me draw class types and property types from no less than six ontologies!

The RDF ontology to describe object properties
The RDFS ontology to describe object classes and labels
The Dublin Core (DC) ontology to describe the titles, authors, and the like
The Friend of a Friend (FOAF) ontology to describe my contact information
The XSD ontology to describe literal dates, strings, and numbers
And yet another, custom, ontology to describe everything else particular to the blog

That is already 6 ontologies, and we haven’t even raised the possibility of using OWL Time, Snap, Span, and GeoOWL for things like time and space description! Even for a semantic web developer, the complexity of managing all of these ontologies, and the namespaces that go with them, becomes pretty burdensome pretty quickly.

And that is why I worry about the future of RDFa in HTML5. It appears that the Microdata specification in HTML5 is essentially the RDF graph data model with the URIs neutered out. Given essentially the same data model, no doubt most developers will pick the easier of two formats to implement.

In order to get more people on the RDF bandwagon, we need to make the RDF path just as easy to follow as the Microdata one. How can this be done? If you ask me, the best way is to get rid of this namespace traffic jam and cultivate a set of community-oriented ontologies.

Rather than trying to create base ontologies that address abstract universal concepts, why not try to have each community standardize a single ontology for their particular domain. Have WordPress and Blogger sponsor the Blog Ontology. Have Amazon.com and eBay sponsor the Marketplace Ontology. Have Facebook and MySpace sponsor the Social Ontology. Then, instead of reusing bits from other ontologies, such as dc:creator or foaf:name, have each of these community-focused ontologies be self-sufficient, covering all the concepts necessary for their domain. We can always apply mapping rules to distinguish between social:name and store:book-author-name later. With only a single ontology per domain area to worry about, the namespace traffic jam will disappear and it will be easier for people to get on board with RDF and RDFa.

All in all, it seems the good news coming out of the HTML5 spec is that we can expect rich data annotation to soon be arriving to HTML content everywhere. But what we need to work on as a community is a way to make URIs, and the Ontologies that give them meaning, easier for programmers to use so that the web won’t just be full of data with Microdata, but full of linked data with RDFa.

Written by Ted

September 21st, 2009 at 1:54 pm

Posted in Computer Science, Haystack, The Interwebs, Web Development

Tagged with HTML5, Microdata, RDFa, semantic web

A More Subtle Security “Hole” in WordPress

without comments

This has come up twice in one day over here at the Haystack group, so I thought I’d post it here.

Using Emacs (and possibly Vi?) to configure Wordpress can expose your database login and password to the world.

Here is why: Wordpress stores this configuration information in a file called wp-config.php that lives in the root of your install. This files contents aren’t visible to web visitors because its .php extension causes to be interpreted by PHP instead of returned to the web user.

Emacs, by default, will save a backup file of anything you edit in the same directory. This file will end with a tilde character (~). This means that if you use a non-customized version of Emacs to edit your wp-config.php file, you’ll get a second file called wp-config.php~. The .php~ extension is not registered to any particular MIME type, and so it will be returned, in full, to the remote user as text, revealing your database server, login, and password with it.

So please, if you use Emacs and have Wordpress, check your WP install directory for any Emacs backup files and delete them! There’s enough Wordpress hackery going around already this week

Written by Ted

September 8th, 2009 at 3:28 pm

Posted in The Interwebs, Web Development

Tagged with Security, Security hole, wordpress

More Six Degrees: An Experimental Study of Search in Global Social Networks

without comments

The followup paper to the prior one I summarized here, also published in Science, re-does the Milgram letter experiments over email. Before going further, here is the citation:

Dodds, P. , Muhamad, R., Watts, D. An Experimental Study of Search in Global Social Networks. Science Vol 301. 8 August 2003.

They take the letter-writing experiments global, ultimately involving over 60,000 people in 13 countries, all trying to route messages to one of 18 targets with only local information. They found the following bits of interesting information:

Friends are routed to far more than other relationship-types
Work ties, School ties, and family ties are the three highest sources of the acquaintance, in decreasing order
People route to “fairly-close” friends more often than very close friends.

They verify that email, as with letters, reaches its target in 5-7 hops using local-knowledge social routing. And they find that routing in this experiment was fairly egalitarian — that is, no one person was critical to the success of the messages reaching targets.

This is an interesting experiment to see how the prior letter-writing one would apply to the digital world. In addition, it collected some useful statistics about the types of people we choose when trying to route information to a target.

Written by Ted

September 8th, 2009 at 12:00 pm

Posted in Computer Science

Tagged with paper, six degrees of separation

Controller-specific TTNavigator URLs with Three20

with 2 comments

ted_iphone

One of the more unique parts of the programming style that comes with the Three20 framework its its URL navigation style via the TTNavigator class. This class is intimately tied in with table cell-based navigation. But what about when you have a URL that is context sensitive. That is, it doesn’t route to a general function, but rather a specific instance.

Here is a scenario like the one I have in my code: I have a “profile” view that allows people to publish various social links about themselves, and one of these is email. When you click the email table cell, I would like it to trigger a message to that controller which opens up the email client:

[self sendEmail];

The problem is that TTTableItem objects want URLs, not delegates and selectors. How do I wire up a URL to this particular controller instance instead of just a general one?

The answer is to stop thinking of TTNavigator routes as something that has to be setup at application start time. I don’t know if you think like this, but I realized I had subconsciously been thinking about routes the Ruby on Rails way — parameters that are only mutable during initialization. But TTNavigator routes can be changed any time.

So here is how I implemented the email link. Each time the PersonController initializes, I have the following code:

TTNavigator* navigator = [TTNavigator navigator];
TTURLMap* map = navigator.URLMap;

[map removeURL:@"app://person/email"]
[map from:@"app://person/email" toViewController:self selector:@selector(sendEmail)];

Notice how I first remove any existing routes, and then I re-wire that route to the particular controller that has just been initialized.

Then when I create my table cell, I can just use a URL as always:

[cells addObject:[TTTableImageItem itemWithText:@"Send Email" imageURL:@"bundle://email.png" URL:@"app://person/email"]];

And that is it! Let me know how you have solved the same problem. As Three20 doesn’t have much in the way of documentation, I would love to hear how other Objective-C hackers are handling situations like this.

Written by Ted

September 4th, 2009 at 6:40 pm

Posted in iPhone Development

Tagged with iPhone, Objective C, Three20

A Formal Model for Travers & Milgram’s Letter Experiments

without comments

Back to school next week, so in my neverending quest to learn how to do Real Research, I’m going to try to begin reading papers regularly. To keep momentum (and to be a better blogger), I’ll write summaries here.

First up is Identity and Search in Social Networks by Watts, D.J.; Dodds, P.S.; Newman, M.E.J. from Science Vol 296, 2002.

They create a formal model to describe social networks which predicts similar path-lengths as Travers and Milgram’s famous letter-writing experiments in the 1960s. For those have haven’t heard, these experiments are where the term “six degrees of separation” come from. The two scientists randomly chose residents in Nebraska and Massachusetts to send a letter to a target person in Boston, and, if they didn’t know the person directly they were to mail it to the person they thought might know them with the same instructions. They found that the average length of the chain from random sender to target was about 6 (and, as an aside, that the probability of termination at any point in the path was 0.25).

Watts, Dodds, and Newman creates a tunable social network model that is able to predict the results of Travers and Milgram, but also resembles many real-world computer science problems. The essence (read: oversimplification) of their model is that individuals are described by (1) connections to friends, and (2) a feature vector of traits, and (3) a tendency to interact with and make social decisions based on hierarchy, where hierarchy is expressed within the feature vector.

Setting up the model as such, it stands to reason that an individual with only local information — that is, knowledge of their own feature vector and the feature vectors of their friends — would route a message to a target T to the friend whose feature vector best matches that of the target. Approaching the world hierarchically, such a friend is most likely to be in an overlapping social group to the target.

They then test their model with a variety of different parameters, with a few interesting findings:

If H, the feature vector, is of too high a dimensionality, the path becomes a random walk. This matches real-world intuition: if we were unable to bucket our social network into groups, we would have trouble deciding which friend to turn to for a particular need.
Searchable networks (those for which senders can locate their targets with relatively short paths) tend to have a positive homophily parameter. Homophily, apparently, is the tendency of individuals to associate with like individuals, as opposed to dissimilar people. This also makes intuitive sense: if you can reliably predict what types of associations a friend has by that friend’s feature-vector, then you can make good guesses as to whether your friend is a good routing choice for a particular target.

And, perhaps most interestingly, when tuned to simulate the conditions of the Travers and Milgram experiments, the model predicts that the average path length between a randomly chosen source and target is 6.7 — right in line with the “six degrees of separation” we all recite.

Takeaways: An interesting model both for its assumptions of how social networks can be formally characterized and how localized routing decisions work. Its routing algorithm produces paths of equal average length to observed human behavior.

Written by Ted

September 4th, 2009 at 2:03 am

Posted in Computer Science

Tagged with Computer Science, graph, paper summary, social networks

Nasty Bug in Rack 1.0

without comments

I spent most of the afternoon tracking down a nasty bug that ended up being in the Ruby Rack middlewear and thought I’d post a fix here in case there are any Rails developers out there who read my blog.

In short, a call to .params on a Rack::Request object will fail with an uncaught exception whenever @env["rack.request.form_input"].eql and @env["rack.input"] are both nil. This causes request.POST to return nil (line 137 of request.rb), which then causes line 160 of request.rb to throw an exception:

160        self.put? ? self.GET : self.GET.update(self.POST)

I wouldn’t be surprised if this bug is already fixed in the latest git repository, but I wanted to post the fix here in case you are on a host where you are frozen at Rack 1.0 and can’t update to the lastest.

Basically, instead of using request.params to access the union of GET and POST params, use the following method:

    def extract_params(req)
      if req.put?
        return req.GET
      else
        (req.GET || {}).update(req.POST || {})
      end
    rescue EOFError => e
      req.GET
    end

That implements the code a bit more safely and will return an empty hash instead of nil. Don’t hate me for using Ruby’s notorious (possibly_nil_var || backup_var) syntax.

Now to finally finish the real work I was trying to get done….I guess the good news is I learned how to use ruby-debug.

I was using Rack::Request to extract the useful data out of the env variable that is presented with the rack request. In certain circumstances, the

Written by Ted

August 12th, 2009 at 11:28 pm

Posted in Ruby, Ruby on Rails, Web Development

Tagged with rack, rails, Ruby

Palm Pre secretly tracks you by GPS, sends info to Palm

without comments

It was bound to happen one day with these new GPS and Internet-enabled phones. First we find out AT&T has been illegally spying on US citizens, now we find out that companies are beginning to do it for their own personal gain.

We dearly need someone on the hill to craft legislation that prevents companies from this type of cyber-stalking, as it can only lead to the types of societies that people don’t want for their kids.

Written by Ted

August 12th, 2009 at 2:30 pm

Posted in Business, Politics

Tagged with Palm Pre

Hi!

I'm Edward Benson, an author, entrepreneur, and PhD student in Cambridge, Massachusetts. This is the site I use to collect my thoughts on computer science and technology in general.

Edward Benson

Hacked By K-E-H

Hacked By Kosova Elite Hackers

Uccello’s Chalice

How Safari and Firefox handle HTML 5 Manifest files

First, what is the manifest?

Firefox

Safari

Conclusion

Will the Namespace Traffic Jam Kill RDFa in HTML5?

A More Subtle Security “Hole” in WordPress

More Six Degrees: An Experimental Study of Search in Global Social Networks

Controller-specific TTNavigator URLs with Three20

A Formal Model for Travers & Milgram’s Letter Experiments

Nasty Bug in Rack 1.0

Palm Pre secretly tracks you by GPS, sends info to Palm

Hi!

Search

View Posts by Category