Archive for CS 462
Book Review: Programming Amazon Web Services
Posted by: | CommentsI don’t know if they’re just a more established tech book publishing company, but I usually have a good experience with O’Reilly books. Programming Amazon Web Services, subtitled S3, EC2, SQS, FPS, and SimpleDB, by James Murty, was great. 5 stars.
I enjoyed this book mainly because I love using Amazon’s web services for recreation and work. If I didn’t enjoy Amazon’s web services in the first place I probably would have found the book excessively detailed. In chapter 5 the author writes, “This chapter delves into the nitty-gritty aspects of running a Linux server in EC2,” and he ain’t kidding! This book really gets down into the API (and this is true for all the services treated in the book, not just EC2).
So if you’re looking to do some casual computing on EC2 or S3, you’d probably be better off without this book. I’d recommend installing the Firefox plugins for EC2 and S3, and going from there. Here’s a link (from the web site of a class I took last fall) that will probably be useful to someone in that situation. On that page you’ll find links to some tutorial pages, and a webcast or two.
On the other hand, if your intention is one of the following:
- Author a tool similar to the Firefox EC2 plugin.
- Create complex scripts to manage your EC2 instances or S3 buckets.
- Write a code library for any of the Amazon web services.
- Increase your understanding of what’s going on when you use the Firefox plugins.
Then this is the book for you.
That said, this book exposed me to FPS and SimpleDB for the first time (never had a chance to use either). As far as EC2, S3, and SQS go… I didn’t really learn how to do anything new with them from this book, per se. But it did significantly increase the depth of my understanding regarding each of these services. There’s a benefit to depth of knowledge with these kinds of technologies, because I’m sure I’ll encounter a problem in the future that can be solved with these tools, whose solution I might have overlooked before.
Final Thoughts – The Labs
Posted by: | CommentsI’d just like to document some of the final thoughts I’ve had about these labs that we’ve done this semester. This is more for the benefit of those taking the class in the future, specifically those who are doing the Amazon EC2 labs.
One thing I did, which has saved me who-knows-how-much time was using a dynamic dns service for my servers. I had one for each of my three major servers (web server, listing server, submit server) which allowed me to hard code my calls during testing. Not only did I not have to refresh as much to get my server, but it also saved me when the submit service load balancer started having problems during the past two (or more?) weeks. Scott Chun has a good description of how to set up your server to register with dyndns.com’s dynamic hostname service. Read about it here.
One thing I would have done differently would be to store those URLs in a config file (elementary 240 stuff) so I’d only have to change it in one place to make the switch from hard-coded to load-balanced.
Something that was mentioned several times in class, and I would still love to see a tutorial on how to do this, was an alternative to frequent image persisting. Especially for minor script changes I didn’t realize I needed until having started the persist process, heaven knows I’ve had more of those than I needed. The solution is to have your server automatically check out the files it needs from a CVS/SVN repository on startup. I’m only an amateur shell scripter, but I assume there are two things the script would need to do. 1) check out the files. 2) make sure they have the correct permissions. Anyway, this would have been a real time saver.
It would have been fun to get a little experience with Pound in the labs. That’s the only thing in the entire process that I don’t feel I could go off and do right now. Sam says it’s pretty simple to figure out, I guess I’ll be finding out in a couple weeks when I’m off on my own to try it.
This class has really been an enjoyable experience. Earlier in the semester when I gave up on Python, I thought I’d feel some remorse at the end for not having stuck with it. Well, I don’t. The architectural concepts the labs have illustrated really do transcend the languages used, and I’m glad I didn’t get so bogged down in the language that I missed the point (that’s the reason for scrapping the EJB labs, right?). One thing I do regret is that Sam isn’t making us use a template for our demonstration for Jeff Barr. Speaking of that, Nathan, you’d have gotten my vote for best design. Did anyone else have a design for consideration?
Lab 5 : PHP and SOAP
Posted by: | CommentsThe only SOAP requests I’ve ever made were made on the .NET platform. They’re not that much of a beast on .NET, but it wasn’t exactly a cake walk either. So I had been bracing myself for the worst trying to implement it in PHP.
I should explain that my lab 5 client connects to a PHP service that in turn makes the SQS requests, etc. I initially wanted to implement an SQS library in Actionscript (and probably will in the future when I’m not pressed by deadlines) but I decided it was too ambitious for the amount of time I wanted to spend on this lab. So alas, a PHP service also handles my SOAP request to WHOIS.
Anyway, I was expecting SOAP on PHP to be a seriously complex affair. Here’s my code that makes the request:
$client = new SOAPClient("http://www.webservicex.net/whois.asmx?WSDL");
$params = array('HostName' => $_GET['url']);
$whois = $client->GetWhoIS($params);
Granted, it would have required about 2 more lines if there wasn’t a URL to the WSDL, but it doesn’t get much simpler than that. I should mention that this requires that PHP SOAP be enabled (uncomment a line in your php.ini if you’re running Windows; recompile from source using ‘enable-soap’ if you’re running Linux). I didn’t have to recompile, thanks once again to Remi Collet (the French guy who has yum rpms for all this stuff, see my previous post).
Well, the SQS library I’m using is pretty old and doesn’t have a means of querying the queue for the number of messages. So, I thought I’d try sending a SOAP message to Amazon to get it. Amazon’s WSDL is a little more complex, and I probably could have gotten it to work if I wanted to play around with the messages for another hour or so. It turned out to be a miserable failure, and I resorted to my old tricks: (file_get_contents()) which worked perfectly. Here’s the code I used, which shows the query string needed to get the number of messages:
$timestamp = gmdate('Y-m-d\TH:i:s\Z');
$qs = "http://queue.amazonaws.com/A3N3IV5XJH079S/processing" .
"?Action=GetQueueAttributes" .
"&Attribute=ApproximateNumberOfMessages" .
"&AWSAccessKeyId=[AMAZON_ACCESS_KEY]" .
"&Version=2007-05-01" .
"&Timestamp=" . urlencode($timestamp) .
"&Signature=" . urlencode(constructSig('GetQueueAttributes' . $timestamp));
$response = file_get_contents($qs);
The constructSig is the same method I listed in a previous post.
Here are a few links that were helpful:
SQS Query and SOAP API
Getting SQS Attributes
SQS WSDL
Lab 4 Schema Clarifications
Posted by: | CommentsI came away from our lab 4 design session with a couple wrong impressions regarding the schema, and got them clarified by Sam yesterday. Here’s a less ambiguous version of the message format we’re supposed to use (would have updated it on the lab page but I’m not on campus at the moment).
<idea guid="">
<initiated date="" technology="(wstechnology)">Name provided by user</initiated>
<submitted date="" technology="(submit server tech)">RY name of submit server creator</submitted>
<spam>true</spam>
<domain>www.foo.com</domain>
<body>Foo and gunk are better for this site than xs</body>
</idea>
You should notice that the wsuser that we POSTed from our submit script is getting thrown away. I assumed that since we went through the trouble of POSTing it, we’d definitely use it– and I ended up putting the user provided name in the submitted element.
Looking back I don’t think we got our schema right. I think we’re trying to cram too much information into too few elements. Sure, it keeps us from having to add an additional element ( for example) to store the information; but the result is that we’ve lost some information– wsuser (less important) and made it more confusing (more important).
SQS: Queue Length / Auth Signature
Posted by: | CommentsTo get the queue length, as well as the visibility timeout, you make a request using the GetQueueAttributes action. The PHP library I’m using to make my calls to SQS doesn’t support this call (must have been written before the 2007-05-01 release of SQS) so my options are to find a new library, or to write my own function to do this.
I decided to try writing my own first, and while researching this I found something I was looking for while doing lab 4. How to compute the authorization header, or Signature.
The process is as follows, you take the query parameters and concatenate them all end to end (key preceding value). Don’t include the ?, &, or = signs. Then you calculate the HMAC-SHA1 signature of that string (using your secret access key). Then convert it to base64.
Here’s the example Amazon gives on their site.
The following request:
?Action=CreateQueue &QueueName=queue2 &AWSAccessKeyId=0A8BDF2G9KCB3ZNKFA82 &SignatureVersion=1 &Expires=2007-01-12T12:00:00Z &Version=2006-04-01
translates into the following string:
ActionCreateQueueAWSAccessKeyId0A8BDF2G9KCB3ZNKFA82Expires2007-01-12T12:00:00ZQueueNamequeue2SignatureVersion1Version2006-04-01
which when hashed with the secret key (fake-secret-key, used in this example) yields:
wlv84EOcHQk800Yq6QHgX4AdJfk= (URL encoded version: wlv84EOcHQk800Yq6QHgX4AdJfk%3D)
I looked at my PHP library, and sure enough here are the methods that create the signature. They require the PEAR Crypt_HMAC package.
function hex2b64($str) {
$raw = '';
for ($i=0; $i < strlen($str); $i+=2) {
$raw .= chr(hexdec(substr($str, $i, 2)));
}
return base64_encode($raw);
}
[/php]
function constructSig($str) {
$hasher =& new Crypt_HMAC($this->secretKey, "sha1");
$signature = $this->hex2b64($hasher->hash($str));
return($signature);
}
[php]
