• valium – A gem that lets you get around the ActiveRecord instantiation penalty while retaining the awesome power of using ActiveRecord.
• crack – A simple XML/JSON parsing library.
• joint – Provides GridFS functionality to MongoMapper… seems kind of like a simple paperclip+fog implementation for MongoMapper.
• cannabis – Simple permissions gem.
• psychedelic – A very “psychedelic”, maybe even just “psycho”, syntax colorizer.
• cocaine – A gem which provides a bunch of functionality for executing system commands.

Can you guess which two we’re using?

I don’t know if these gems are a positive or negative for the Ruby community. Personally, I like that the ruby community has such a great sense of humor. There are other humorously named gems out there that have nothing to do with drugs, like polyamorous and bourbon. I think it’s fantastic the Ruby community seems to remain constantly aware of the fact that programming should be fun, artistic, and expressive. My initial reaction is that these “funny” gems are just a symptom of that awareness; behavior that is generally embraced by the community.

What do you think? Are these gems good or bad for the community? Any other drug themed or humorously named gems I missed? What about libraries from other languages?

The unix/linux grep command can use POSIX extended regular expressions. I learned from this guide that unlike most other regular expression implementations the meta-characters must be escaped to be treated as meta-characters. If they are not escaped grep will treat them as literal characters. This is exactly opposite of most regular expression engines where you escape meta-characters when you don’t want them treated by the regex engine as special characters.

I’m not particularly great at regex and I didn’t find a lot of guides on using extended POSIX regular expressions with grep. This really surprised me.

In my scenario I had a really large file I needed to search through. I had several unique strings I thought might be in this file and due to its size I didn’t want to execute a separate grep command for each string. This would be woefully inefficient as the entire file would be searched independently for each string. This approach would also take much much longer to complete and would be a lot more work for me.

Instead, I knew there had to be a way to craft a single command that would search the file once looking for any lines containing any of the strings I supplied. Well, using my mediocre knowledge of regular expressions and my new-found understanding of the behavior of grep’s regex engine I came up with the following command:

Breaking down the command further:

• grep – Indicates we are using the unix/linux grep utility.
• -nE – These are the grep “options.” ‘n’ tells grep to tell us which line number any matches came from when it spits out its results. ‘E’ tells grep to use its extended regular expression engine (required for this kind of search).
• \( and \) – These are the “range” boundaries for the regular expression. Technically not required here, but I find it good practice to use ranges in case you ever need to back reference. Grep would treat these like any other string if they weren’t escaped with the backslash. We want grep to treat these as special regex characters (range boundaries in this case) so we must escape them.
• \| – The regex “OR” operator. This tells grep that it can match either the value to the left or the value to the right of this operator. If you read out the statement above as “or” the command makes perfect sense. We are telling grep: find string1 OR string2 OR string3 OR string4 in theFile.txt. Since we want this to be treated by grep as an OR operator and not just another character it also needs to be escaped with a backslash.
• string1, string2, string3, and string4 – The strings grep should be looking for in the file. The line “I like the taste of string1′s bread” would match string1 and would be in the results outputted by grep. The line “I think String2 has a really mean demeanor” would not match since this search is case sensitive by default.

• humble little ruby book [PDF] (from humblelittlerubybook.com)
• WikiBooks: Ruby Programming – WikiBooks is a WikiMedia project and is similar to WikiPedia; however, content is organized under specific topics and meant to be treated somewhat like a book. I created a collection of all the Ruby Programming pages I could file and added in the print version of the Ruby on Rails topic as well. I then used the book creator to export the collection as a PDF. I am hosting a snapshot of this collection from 4/9/2010 here [PDF].
• The Little Book of Ruby [PDF] (from SapphireSteel Software)

I am looking at solutions for cleanly converting the RDoc documentation for ruby core, Rails, and a variety of gems. I will update this post or make a new post if I figure that out.

What I learned about while I waited was the *nix command ‘sed’. This is a tool built for command line processing of data files. Apparently it was birthed as an evolution of our trusty friend ‘grep’. The forum post I found which hinted that ‘sed’ was my solution didn’t provide much real information and the Wikipedia article was mostly background and provided examples that won’t help me.

Where I found the most useful info was the sed page on sourceforge… go figure. The docs page pointed me to ‘The sed one-liners‘ by Eric Pement. Here I found, through example, the power of ‘sed’ and an example that is more efficient on large files than the ones I found elsewhere.

So here is how you do it:

’34005050′ is the line number. ‘q’ tells sed you are looking for that line number, and ‘;d’ tells it to stop after that line. ‘filename’ is of course the file you are trying to coax a specific line out of. To do an inclusive range of lines all at once (lines 8 through 12, for example), do this:

I’m still learning about ‘sed’ but its already saving my ass. Have fun.

I decided to use this as a chance to learn much more about building secure web applications and I spent a few hours diagramming the concept. Then, I spoke with my friend again. I explained the design to her and she agreed the design would keep her journal secure enough that she would feel comfortable testing it and using it.

The basic design of ExaNotes is that the a user’s password is used to encrypt each user’s notes using the very strong AES-256 cipher. This password is only available to ExaNotes during the duration of the user’s browsing session. Once a logout or timeout has occurred the session data is destroyed and there is no longer any information left on the server which could be used to decrypt a user’s notes. The only thing stored on the server are the salted SHA-256 cryptographic hashes of the user’s username and password. Even the username is anonymized!

In reality there is no way to completely secure any web application from someone who has all the keys and the knowledge. A sufficiently skilled developer or administrator could modify ExaNotes in such a way that user credentials were captured at the time of login and stored somewhere, allowing them the ability to decrypt the notes of users whose logins they have captured. As it currently stands ExaNotes only retains the login credentials for the duration of the user’s session and the session data is not easily accessible. More than just my good nature allows me to promise ExaNotes will remain this way as long as I am administrating it. My own self interest is to thank as well. If I had access to the content of ExaNotes’ users I would assume a level of liability I do not want. Building a way to gain access to user content could force me to reveal content to a 3rd party. If the 3rd party was a good intentioned police officer wanting access to content to solve or prevent a murder there is very little ethical dilemma from my perspective. However, if an agent of an oppressive political organization attempted to coerce or force me into providing access to a user’s notes in an attempt to track and suppress political dessent it would be nice if I could avoid the ethical quandary by being able to truthfully respond that I simply have no way to provide access. I feel preventing myself from assuming that liability and building a truly secure and anonymous system is even more important than providing any mechanism for 3rd party investigation.

For some my promise is understandably not enough. To allow for independent validation of ExaNote’s design and security and to allow people to run ExaNotes on their own server I am offering the source code to anyone who requests it. For now anyone interested can send me an email and I will send you the source. Eventually I plan to start an open source project and release the source code under an open source license. I simply have not found the time and have not yet had requests from other developers to do so. For those interested ExaNotes is built using PHP,MySQL,HTML,CSS, and JavaScript. ExaNotes makes use of the prototype javascript library and tinyMCE rich text editor for its web interface. ExaNotes also contains a small API which could be used for desktop or “native” clients. If there is interest the API will be published with documentation.

The user interface of ExaNotes is clean, simple, and easy to learn. I generally follow the KISS principle in my interface design, which stands for “Keep It Simple, Stupid!” I’m sure you will agree that I have done that with ExaNotes. While the interface is simple I do not believe it is boring. ExaNotes provides a slick, even elegant interface for writing and managing your notes. This interface is lightweight yet modern, making use of modern web development tools like Ajax and Rich Text Editing while running fast on even the slowest computers and the smallest screens. ExaNotes even runs well on some mobile devices and is particularly optimized for use with the Palm Pre.

ExaNotes has been functional for a few months now and has been my primary note taking tool for roughly as long. I have been developing ExaNotes to fit my needs and those of a few of my friends and colleagues. I now feel this application is mature enough that it is ready for a larger audience and hopefully there is a large audience out there who will find it as useful as I do.

The public beta begins today. Simply visit ExaNotes to participate.

Please send me questions, criticism, suggestions, bugs, or any other feedback you have regarding ExaNotes. Leave a comment here, send me an email, message me on twitter, even find me on AIM or Facebook.

Like last time, this script is also available through userscripts.org.

I haven’t tested this in Greasemetal (Google Chrome) or Greasekit (Safari) yet, but it uses very simple and standard javascript DOM to work, so I’m pretty sure it will work fine in other userscript environments.

Pretty simple, but its something I really wanted so I figured I would share in case anyone else needs a tool like this. Also available through userscripts.org.

Let me know if you can think of any other sites this would be useful for.

Output Buffering example:

Single Quotes example:

Double Quotes example:

All of these methods should build a variable $string with content similar to the following:

To determine how much slower output buffering is than the other two methods I wrote a benchmark (source code) which repeats each method a large number of times. The benchmark linked to here is hosted on a shared hosting server so the results are unlikely to be consistent. I used an Athlon XP 3200+ system with 1GB of RAM running Apache 2.2.8 and PHP 5.2.6 on Ubuntu 8.04. I made sure that the system was not doing anything but serving up this benchmark. The results were repeatable and consistent, and the following is an average of five separate benchmark runs.

As you can see, output buffering is clearly the slowest method. What is surprising, however, is that the difference is fairly small. The greatest delta exists between Output Buffering and Double Quotes, which just tops 16% when the tests are ran in consecutive mode. Even 16%, though, is not a huge performance difference. Its still small enough that using output buffering exclusively on a larger project for building large strings may not be considered a significant performance impact, and the advantages of using output buffering may make it the obvious choice.

Next, I decided to see if using logic while building strings would have any affect on the performance overhead output buffering incurs. In case what I am saying is not clear, here are a couple examples of using logic while building a string using both Output Buffering and Single Quotes:

These examples further highlight the advantages of using Output Buffering. The code is clearly cleaner and easier to work with. With traditional string building (using single quotes, double quotes, or even heredoc) the string must be terminated, the conditional evaluated, possibly concatenating the string, then the string must be concatenated a final time to add additional closing string data. This is convoluted and ugly, and the performance difference would need to be immense for me to consider building strings like this using single quotes or double quotes. However, due to the additional string concatenation my hypothesis was that adding logic would actually lessen the performance overhead when using Output Buffering, making Output Buffering even more attractive. Lets have a look at what the actual results show when we add logic:

While I expected a reduction in the performance cost of output buffering, I was not anticipating a reduction of this magnitude which such a small amount of logic. Against single quotes, Output Buffering has reduced its performance disadvantage by approximately 2/3, and compared to double quotes the disadvantage has been reduced by well over 1/2! These results are very encouraging for those wishing to use Output Buffering. The results from adding a small amount of logic lead me to believe that the performance disadvantage of Output Buffering will continue to shrink as the size of the string and amount of logic increases. In fact, a large enough string or frequent enough use of logic may eliminate the performance disadvantage completely and could even give output buffering a performance advantage in certain situations.

In the end the most important part of this investigation for me is that I will be much more likely to use output buffering in the future.