Monday, June 30, 2008

The SPARQLTaggingServlet with ED

Now that the SPARQLTaggingServlet and SPARQLTagManager are complete, I've gone ahead and integrated them into the rest of ED. I call the SPARQLTaggingServlet as I would ED's current TagManagerSaveTaggingServlet. EXCEPT! I stop the form from submitting to Connotea until it's confirmed that all the tags have first been put into our Virtuoso database. Basically this involves having to 'return false' on all the functions bound to 'submit' so that the form doesn't submit. and then setting a setTimeout to check for a response from SPARQLTaggingServlet saying it's okay to submit before calling submit from the javascript.

Interesting point: Ben discovered an error in this new addition. When he tried adding a new bookmark, Tomcat went crazy saying it was low on memory. Although I did some basic testing on it, I guess I didn't do enough... I suspect that it's because I forgot to close something that needs to be closed.

Thursday, June 26, 2008


I've added in 1 new class and 1 new servlet to the org.icapture.ED.tag package. is a servlet that functions in the same way as is a java class that functions in the same way as the

When a user hits the submit button on ED - the javascript normally calls the TagManagerSaveTaggingServlet (passing into it all the tagging data in xml format as a parameter). SPARQLTaggingServlet will be used in the same way. Call it in the same way with the same xml. SPARQLTaggingServlet acts as Servlet wrapper for the main class which is SPARQLTagManager. It creates a Tagging object with that xml and passes it into SPARQLTagManager which processes it into SPARQL/Update (SPARUL) INSERT statements and posts them to a Virtuoso server via "" as parameters which are then executed.

If it is successful (determined inside SPARQLTagManager via response codes), all inserts were made successfully and it will return the ResponseCode from the Virtuoso Server (200 if successful, anything else otherwise) to the SPARQLTaggingServlet.

Tuesday, June 24, 2008

Manual URI entry Form for ED!

Forgot to post about this earlier.
Did up a manual URI form for Entity Describer so that you can enter a URI manually.

URL being ''

Posts the URI to Connotea and ED Database as well and requires that you enter a URI and Title.

Problems I ran into:
Not sure how to circumvent Connotea's fixsize() function which resizes your browser window. I can see that becoming really annoying in the future.

Any ideas?

Monday, June 16, 2008

Hash instead of SessionID

With regards to security of ED, Ben suggested we drop the sessionID cookie because it places a need on the server to keep that sessionid (which will eventually expire). Since what we want is to be able to log into ED and stay logged into ED indefinately by relying solely on the cookie that we pass onto the client for authetication. (This may prove troublesome for people who decide to use computers other than their own and don't log out - effectively not deleting the cookie either).

Instead, I've gone ahead and had a look at how Connotea does it using a Hash value in a cookie to use for authentication. To learn more on it I googled "hash" and "cookie". Several interesting recipes later... I decided to refine my search. Just another example of how big a difference context and semantics make in the web!

Lucky me since Java libraries already provide me with a means of generating Hash values via the MessageDigest class. I generate a hash value using the user's openid, logintime and a secret value known only on the server's end. I then create an ed_hash_cookie for the user which contains the openid, logintime and hashvalue.

To authenticate for a restricted page, we look to see if the ed_hash_cookie exists. and then recaculate the hash_value from the cookie's openid, logintime and our secret value to make sure it matches the hash_value in the cookie.

This will make it harder unauthorized users to use ED since it requires them to make a cookie with a username, logintime and hash value. Unless they know the secret value used by ED to calculate hash values, it will fail the authentication when the server re-calucates the hash and finds a mismatch with the cookie's hash value.

Of course, this is still vulnerable to the same attacks as described earlier regarding cookie theft. Someone could still potentially sniff out a hash_cookie (er... no pun intended) and use that to get into ED.

I'm using the MD5 algorithm currently to calculate the hash and encoding in Hex, but I'm going to switch to SHA given that SHA is the successor to MD5 and used in TLS,SSL and other security applications.

Thursday, June 12, 2008


Note to self: ED 2 Connotea uses existing javascript from Connotea. ie. the addtag(tag ,clear) function

For EDnoConnotea (ED seperated from Connotea) I've created a javascript 'copiedFromConnotea.js' with... as you might guess, javascript copied from Connotea.

Will remove superfluous functions later.

Wednesday, June 11, 2008

OpenID Providers

This is a list of the OpenID Providers I've been using for testing:



For Connotea: I've noticed most OpenID Providers work, although Yahoo, myVidoop, and claimID seem to give it problems. This may be because it doesn't know how to deal with https://

So far the only ones I've had trouble with are Vidoop, which may be (as Ben suggested) because of a missing parameter.

June 16, 2008

Figured out what was wrong. bracket. Wrong place. Funny how still let me through. It's probably because they're still operating off version 1.0 of OpenID and not 2.0, which is where my mistake occured. But praises to Vidoop for replying to me!

Tuesday, June 10, 2008

ED, Connotea, OpenID

I've added in a link to for the current ED to login with an OpenID.
Really all it's doing is submitting the OpenID to Connotea to handle the rest of the work.
But the trouble is, once I've passed it off to Connotea, it's out of my control, so I can't have the same window/tab redirect back to ED once it's done authenticating.

So what I've done is I have is the AddToED servlet opening in a new tab/window while the other one handles the authentication.

I imagine this will lead to confusion later if the authentication doesn't go through since people will simply continue on with ED and THEN find out they haven't actually logged into Connotea once they've submitted their bookmark and annotations.

Another issue I ran into with Connotea, was that it doesn't like all OpenID Providers. I've benn trying the more common providers: Yahoo, Vidoop and myOpenID, but so far the only one that has worked is myOpenID.

And yes... I did make sure to change the OpenID in the Advanced Setting of Connotea before trying each one.

Monday, June 9, 2008

About FetchRequest of OpenID

With regards to an earlier post:

Using a FetchRequest:
This is how it's done in version 2.0 and this is what all OpenID Providers use. Currently I haven't got this working yet, but as I found out, it was because I don't have the lastest version of OpenID4Java and will have to get the latest from their SVN off of googleCode. (I'll update this as soon as I get it working) This googlegroup post explains it in more detail.

I checked out the latest Openid4java off of their SVN on GoogleCode at: openid4java-read-only

Went into my console, went to the project I just checked out and typed in:

>ant jar

to build a new jar for java-openid-sxip. The jar will be located in the build folder as: java-openid-sxip. I replaced the existing java-openid-sxip jar in my referenced libraries and as a result, it has fixed the earlier problem I was having with the FetchRequest not working. This was because my parameters were showing up:

key: openid.ns.ext1

when it should have been just

key: openid.ns.ext1

The -draft7 that sxip left in there as a download prevents the FetchRequest from working. The only trouble I'm running into now is Yahoo! identifying my website as unsafe and not sending it user information. This is most likely because I'm running the webapp locally under localhost, which is not a valid url for openID to Yahoo! and thus considers it an unsafe website.

Sunday, June 8, 2008

Entity Describer and GoogleCode

Ben suggested putting Entity Describer up on GoogleCode to make it a publicly available tool (which technically it is right now)... But on GoogleCode people will be able to contribute stuff to it! and as Ben put it, "It'll make sure we keep good coding style, since other people will be looking at and using it".

I've had previous projects on GoogleCode because it makes source control for school group projects so much easier, but I couldn't remember if there were any rules ED should be concerned about, so I had another quick look.

-Anything that goes on GoogleCode is Open Source.
-We can terminate the project whenever we wish to and take it off GoogleCode
-GoogleCode can terminate our project if they catch us doing something illegal
-When you create a project, you choose an open source license (
GNU General Public License v3 , Apache License 2.0)
-On set up you establish the project owners and the project members who are both capable of contributing/making changes to the code
-Anyone may download a READ-ONLY copy of the code (ie. Can't commit changes to GoogleCode for this specific project)

Looks good to me.

Terms of Service

Friday, June 6, 2008

Trying out using OpenID to fetch user information

OpenID lets you log into an existing account on a website using an OpenID URL. It can also be used to register you with a site that you have NO account with. Normally when you sign up for a site you have to create an account with profile/persona information. UserName, Password, email address, Firstname, lastname, DOB, etc etc etc. However, using an OpenID URL on a site you've never been to automates this process.

This is what happens:
-The site (aka Relying Party) recognizes you're new and don't have an account with them.

-The site contacts the OpenID Provider asking them to authenticate you, it adds on an extension asking for your information: UserName, email, address, Firstname, lastname, DOB, etc etc etc.

-Your OpenID Provider SHOULD prompt you for permission to give this information to the site and lists out the information the site wants.

-You click "Yes" and the site automatically creates an account for you using that information.

Because ED may very well have a user management system in the future and want to keep to information on its users ie. e-mail. I've left hooks in the OpenIDHandler to do this. Some things you may want to know about:

There are two different ways of fetching user information from an OpenID Provider.

Using a SReqRequest:
This was how it was done during OpenID version 1.0 and a lot of providers still allow it such as: Vidoop and myopenid, but other ones like Yahoo do not.

Using a FetchRequest:
This is how it's done in version 2.0 and this is what all OpenID Providers use. Currently I haven't got this working yet, but as I found out, it was because I don't have the lastest version of OpenID4Java and will have to get the latest from their SVN off of googleCode. (I'll update this as soon as I get it working) This googlegroup post explains it in more detail.

What's the difference?
FetchRequest gives you a few more functions to get more information if you want. Such as being able to get more than just one e-mail address.

I've put in both as hooks and for testing I've put both in a different if statement. Activating and deactivating them depend on the values in org.icapture.ED.openID.Constants

Thursday, June 5, 2008

OpenID, Entity Describer and Security

Originally I wanted to publish restricted sites under the web.xml and take advantage of HTTP's existing authorization and authentication functions. This way 401 or 403 errors would be produced on trying to type in restricted URLs without authentication. However I have not found a way to use these basic authentication schemes (Basic, Form-based, digest and Client Certificate) without implementing a username and password and leaving all the authentication work to be done by these schemes even though authentication is already being done by OpenID.

The next method I looked at to restrict site access was using the SessionID. So as it stands, as soon as ED receives the the authentication from the OpenID Provider and ED has verified it, ED sets a sessionID cookie for the user. So long as that cookie exists, the user may access ED without re-logging in. On logging out, the sessionID cookie is destroyed and the session is invalidated. This method requires that at every restricted page there be a check to determine if cookie sessionID from the client and the sessionID from the server match. If the session is invalid or the cookie sessionID does not match, the user is redirected to the login page.

Cookie Theft
The major concern with keeping a cookie for the sessionID is that a person may determine the sessionID from packet sniffing, leading to possible impersonation of a user. One way around this is to use SSL or TLS which encrypts data going back and forth between the client and server, thus encrypting the cookie. This service is requested of the Web Server which hosts your website.

If anyone has any other thoughts on how to implement security, I would be more than happy to hear it.

OpenID and Entity Describer

So in the last week, I've gone about the task of setting up and testing openID to for the next Entity Describer minus Connotea. Fortunately there was a lot of forum posts and advice on the web on how to get OpenID set up for a website.

A good overview of it is: A recipe for enabling OpenID on your site
Since Entity Describer is programmed using Java Servlets, I used the OpenID4Java library. This one in particular because it specifically state version 2.0 which is what most OpenID Providers conform to now. However, for other programming languages a complete list of libraries may be found here.

After installing the libraries, the rest was implementing the servlet to handle the openID login. I've saved it under the ED project as package org.icapture.ED.openID as Most of the example code I was following came from the library I downloaded from OpenID4Java. Particularly under the INSTALL file and the in the examples folder.

The workflow for OpenID goes as such:
-User requests the OpenIDHandler Servlet and a ConsumerManager object is instantiated. The ConsumerManager is the main OpenID object which handles most of the communication between ED and the OpenIDProvider(OP).

-The user submits an openID URL eg. (

-Our website (also referred to as a Relying party or Consumer) extracts from that who the OpenID Provider (OP) is and does a discovery on them to make sure they exist. This is done by calling the discover function of the ConsumerManager.

-Once our website knows the OP exists it forms an AuthRequest message to request Authorization from the OpenID Provider. This is done by calling the ConsumerManager's authenticate function. This will cause the OP to do a GET on the returnToURL you passed into the authenticate function. With ED I gave it the OpenIDHandler's URL with the parameter "?return=1". Bare with me, this is where it gets interesting because I'm going to start talking about XRDS documents and YADIS protocol

-When the OP is doing a GET on the returnToURL, it is expecting to get an XRDS document from it. What's an XRDS document? it stands for
eXtensible Resource DescriptorS. It's also known as a Yadis Resource Descriptor. In it I've specified the returnToURLs OpenID may return to after it's done authenticating (I just tell it to go back to the OpenIDHandler servlet). It's used to let the OpenID Provider know that the returnToURLs are using openID.

-There are a few different ways of giving the xrds document. But according to the Yadis Protocol you should do this by adding the header "X-XRDS-Location" to the response with the value being the URL where your xrds document is located.

response.addHeader("X-XRDS-Location", xrdf_path);

In ED I've named the document: Yadis_xrds_doc.xrdf in /WebRoot/ed. I've also added the mime type for .xrdf extension under the web.xml


-*Please note if you put a breakpoint during this GET. It will cause the authentication to fail.

-After all that the OP has now verified that you're OpenID enabled and you've got an AuthRequest object. What to do next? That depends on the version of OpenID the OP is using. Usually it's version 2.0, but ED handles both version just in case. If the OP's version 1.0, just redirect to the OP's URL that's responsible for handling authentication (also referred to as OPEndpoint) like so...


If it's version 2.0 then it's a little more complicated. Basically what happens is you add 2 attributes to the HTTPrequest. One being the OPEndpoint and the other being the parameterMap you get from the AuthRequest. Then you forward the request and response to a dispatcher. A dispatcher being just another page that uses the same request and response and does the work. With openID version 2.0 the dispatcher MUST have a form with a sumbit button and the parameterMap is turned into inputs of type HIDDEN inside the form. The form posts to the OPEndpoint.

-The OP handles the authentication from here and returns control back to the URL designated as the return to URL. Here, ED verifies the Response from the OP by calling ConsumerManager.verify. This is to make sure it DID come from the OP we sent to. If it is verified, the user is authenticated and ED changes state to logged in.

Useful Links:
OpenID4Java JavaDocs

OpenID version2.0 Specs

Brief Description of what YADIS is

Wednesday, June 4, 2008

hello world!

Hello fellow ED developers! 

Introduction to openID

The rundown:

What is it?
OpenID lets you log into (or even register with) other websites that supports openID by just using an openID URL (eg. instead of a username and password. This does away with the need for having a seperate identity (username and password) for each website that requires you to register with them. Websites supporting OpenID will usually have a link to a openid login page below their normal username/password login page. You get an openID URL by registering with an OpenID Provider such as yahoo or myopenid.

With openID, a website (the relying party or "consumer") delegates the task of authentication to your OpenID Provider (the website that you have an openID with). You can even register with a website you don't have an account with since the openID Provider gives them the registration information that website needs, such as address, e-mail or full name.

Example OpenID Providers:

The official OpenID website
Specs on OpenID version 2.0: Good for definitions