Building a mobile agenda for JSConf

[Update: Sorry for the missing pics... Deleted by accident. Thank you google...]

A couple of OutSystems' engineering guys were going to a JSConf Europe conference this weekend. While looking for the schedule, they found this:

 

We just couldn't let this opportunity go by, so João Neves used Platform 9 (which is about to be released) to build an agenda for the event. You can find the application here. It gets the Agenda from google docs using the new REST capabilities, and it is a completely mobile ready!

And, since on the week before I was playing with the source code of OutSystems Now, I built an application for Android that would allow attendees to view the schedule of the conference on their devices, even if offline (source code here).

The results were pretty cool! At the time of writing, 125 people had installed the Android App and we a bit over 7.500 pageviews! And we had some good reviews! :)

    

Circular Menu with D3.js

WheelMenu is a circular menu aimed at mobile that allows you to quickly select from a small set of options quickly and with your thumb. You can see an example of the WheelMenu running in jsFiddle.
Right now the menu only works for 6 slices, but let me know if you need other number of slices.

Check the github page for more details on how to use the WheelMenu and for access to the source code. If you're using the OutSystems Platform, head over to the community and download the component.

Credits to the WheelMenu original design go to Tiago Simões and Daniel Nobre.

D3 drag and drop tree

I needed a heavily customizable drag and drop tree and I couldn't find one that really fit my needs, so I built one. It was also a great excuse to learn about D3, although most of the work was messing with the quirks of drag and drop.

Anyway, although the code is far from decent and the tree is still work under progress, I decided to make it available to whomever wants to use. I hope it's useful for you, and be sure to let me know if you have any problems.

More info and download at github!

Automatically building a cache manifest file

HTML5 has the capability to work offline and it allows you to cache files of your application on the browser by using a cache manifest. The question is, which files to cache?

Simple applications, with an HTML file, a couple of JS and CSS files are not a problem. But as soon as you move to more complex applications, the number of files explodes - think imported CSS files, background images, images within your page, etc. So here's a little help to build your cache file!

1. Start by getting the offlineHelpers.js file and put it on your website. It's easier if you put it on the folder of the page you want to cache, but you can put it anywhere. If you don't have a site yet, download the whole project and play with the provided test.html file.

2. Be sure the page you want to cache includes jQuery. This is only needed to run the offlineHelpers script, you can remove it afterwards. You don't even need to download it, just src the script from here: http://code.jquery.com/jquery-1.10.1.min.js

3. Launch chrome (yes, for now it only works on chrome). Load the offllineHelpers script by opening the console (cmd+alt+j on the Mac) and typing $.getScript('offlineHelpers.js'). Next, call offlineHelpers.getCacheFile() on the console.

4. Calling getCacheFile should result on a bunch of debug lines (in gray) followed by your cache manifest file. Something like:

Getting stylesheets
Found an import: jumbotron-responsive.css
Getting JS files
Getting Image files
CACHE MANIFEST

# CSS Files
css/bootstrap.css
img/glyphicons-halflings.png
img/glyphicons-halflings-white.png
css/jumbotron-narrow.css
css/jumbotron-responsive.css

# Javascript Files
http://code.jquery.com/jquery-1.10.1.min.js

# Image Files
img/creatures.jpg

Next, tweak the result and copy to your cache.manifest file.

Let me know how this worked for you! If the script missed a file, let me know on the comments and I'll try to fix it!

Why developers suck at UI design

Yesterday I was listening to this podcast from Boagworld  dedicated to the dangers of "what if" design. The basic idea is that you shouldn't let exceptions hinder the experience of the majority. I.e. if something happens once in a blue moon, you shouldn't compromise your UI design because of it.

At a certain point he alerts for the fact that developers tend to make this mistake because developers are worried about edge cases - "what if the connection to the database is broken", "what if the user isn't logged in". I agree with this, and I believe this is one of the reasons developers tend to make interfaces that are overly complicated for the common use case.

The solution seems easy, right? Developers need to take a step back from their work, look at the user-stories they captured from their users, make the most used features really obvious, and hide the rest of the clutter.

Well, nothing fundamentally wrong with this approach. I believe developers must deliver the best possible interface they can, and that requires effort and training. But I don't believe this is enough.

The problem is, when you dive as deep as a developer does on the project, it's very hard to step back and take a look at the big picture. What developers really need to do is get someone from the outside to spot the obvious errors that are just too evident to be seen by someone buried in the nitty gritty details of a project.

Again, this sounds like a simple enough solution. After all, getting someone to comment on your work isn't really hard, is it? Specially if it's about design! But the problem is that most developers don't know how to listen or how to act on the feedback they receive. It's the "what if" problem again... it's really easy to dismiss feedback based on edge conditions that happen 1% of the time. (This makes a great topic for another blog post, and I urge you to write it!)

In the meantime, no time like the present to become better at UX design. Here's an interesting webinar with "6 steps to engineering awesome user interfaces". Have fun!
(disclaimer: this is a webinar recorded by me for OutSystems, the company I work for)

Setting up Cake on a custom folder

I wanted to try the PHP framework Cake, but if you look at the documentation it assumes you want to install Cake under the /var/www/html folder.

You can obviously install to other folders, but if not done properly, you'll encounter a start page with no images, no styles, and some error messages.

The steps I took to make Cake work in my folder (/home/me/dev/myapp) and to work with the http://localhost/myapp URL where:

1. Uncompress the latest version of Cake to /home/me/dev/myapp
2. Edit my /etc/apache2/users/me.conf and add the following (replace braces with chevrons):

   Alias /myapp/ "/home/me/dev/myapp/"
   {Directory  "/home/me/dev/myapp/"}
    AllowOverride All
    Order allow,deny
    Allow from all
   {/Directory}

3. Edit the .htaccess file in your /home/me/dev/myapp folder and add RewriteBase /myapp/ to it. Should look something like (replace braces with chevrons):

   {IfModule mod_rewrite.c}
    RewriteEngine on
    RewriteBase /myapp/
    RewriteRule ^$ app/webroot/     [L]
    RewriteRule (.*) app/webroot/$1 [L]
   {/IfModule}

4. Make sure your /home/me/dev/myapp/app/tmp folder is writable by all
5. Restart apache

And that's it. After these steps you should be able to access your application and see the default Cake page, including images & CSS.

Hello world in Cocoa-Ruby

Here's a short video on how to do a "hello world" application in Cocoa-Ruby.

This was based on this japanese video I found on the web.

PHP on the Mac

If you have a Mac with Leopard and want to use PHP, most of what you need is already installed. It's just a matter of making it work.

First you need to turn on apache. Just go to your "System Preferences", "Sharing" and turn on "Web Sharing". That's it, apache is running. You can check if all is ok by going to http://localhost.

You'll probably want MySql too. Just go to their site, download and install. No big fuss, but make sure you install the StartupItem if you want MySql to start when the system boots.

Next you must enable PHP. Launch a terminal and use your favorite text editor to edit /etc/apache2/httpd.conf. Look for LoadModule php5_module and uncomment the line. Restart apache with sudo apachectl graceful.

Now let's say your username is john, and you have on your home folder a ~/dev/phpproj where your php code is. One way to make this available (and there's lots of others) is to edit /etc/apache2/users/john.conf and add the following to the bottom of the file (replace [ with "lesser than" and ] with "greater than". Blogger doesn't like those symbols):

Alias /phpproj/ "/Users/john/dev/phpproj/"
[Directory "/Users/john/dev/phpproj/"]
 Options Indexes MultiViews
 Order allow,deny
 Allow from all
[/Directory]

Now you can access http://localhost/phpproj/ to view your app.

At this point you might get a MySql error stating that you /var/mysql/mysql.sock does not exist, just go back to your terminal window and type:

cd /var

sudo mkdir mysql

cd mysql

sudo ln -s /private/tmp/mysql.sock mysql.sock

That's it!

Registry entry for Java Preferences

Java Preferences in Windows are stored in the registry entry:

HKLM\Software\Javasoft\Prefs\

and

HKCU\Software\Javasoft\Prefs\

There, now I don't have to dig this information on the net again!

Creating an SVN repositoy

I do a bunch of personal projects and, in the spirit of keeping it all under source control, I always create an SVN repository for all of them (even if I don’t finish them… after all, I suffer from NADD). Because I like to automate everything, here’s the script I use to startup projects:

#!/bin/sh
TMPDIR=/tmp/svntmpdir
REPODIR=/home/rsc/Documents/repositories
REPOURL=file://$REPODIR

if [ $1 == ""] ; then
    echo Usage: $0 {repository name}
    exit 1
fi

echo Creating directory structure
echo $TMPDIR
mkdir $TMPDIR
mkdir $TMPDIR/trunk
mkdir $TMPDIR/branches
mkdir $TMPDIR/tags
echo Creating repository $1
svnadmin create $REPODIR/$1
cd $TMPDIR
svn import $REPOURL/$1 -m "Initial structure"
echo Checking out
cd -
svn co $REPOURL/$1/trunk $1
echo Cleaning up
rm -Rf $TMPDIR
echo Done!

MySQL fails to start on Gutsy

For some reason, mySql wasn't starting up on my machine. It was easy enough to find the cause just by looking at the log. Apparently there was no /var/run/mysqld directory where mySql could write to. So I created the directory, gave the required permissions and didn't give it a second thought.

Turns out that when I rebooted my computer, mySQL wasn't starting again! The directory wasn't there anymore! Because I don't want to have to create the directory and start mySQL by hand each time the computer restart, here's what I did:

sudo gvim /etc/init.d/mysql

Just after the 'start') string I added the following content:

'start')
  # Create directory in /var/run
  if [ ! -d "/var/run/mysqld" ]; then
      mkdir /var/run/mysqld
      chown mysql.mysql /var/run/mysqld
  fi
  # All done

Greasemonkey tips

Greasemonkey is a great Firefox add-on that allows you to add custom JavaScript to any web page. This simple concept allows you to do extraordinary changes to a web page and you can find a lot of pre-made scripts around. But if you want to do your own scripts, there's a few things that you should know before you start...

Registering events
Let's say you want to catch a button click. Here's how to do it:

var button = document.getElementById(buttonId);
button.addEventListener('click', myFunction, false);

Don't do stuff like button.onclick = "myFunction()" or even button.onclick = myFunction. It won't work.

Accessing element properties
Suppose you want to add some information to a link that you created to use during the onclick event. You must do it with:

lnk.setAttribute("attrName", value);

and fetch it with:

lnk.getAttribute("attrName");

Doing lnk.attrName or lnk["attrName"] will not work, and will return null when you access it.

Parameters in Event Handlers
Let's say you registered an event like mentioned above. The event always receives a MouseEvent argument, so you should declare the function like myFunction(evt). This is important because if you by accident register as an event handler a function that takes a boolean paramenter, like myFunction(flag), flag will always be "true" when called from the event.

Stopping the bubble
Assuming you defined your function like mentioned earlier, myFunction(evt), you can stop the event from bubbling by calling

evt.preventDefault();

Testing functions
Because Firebug is such a great tool to debug your scripts, you may want to check for Grasemonkey functions before you use them. Do it with something like:

if (typeof(GM_getValue) != "undefined")

And you should be able to run the script outside Greasemonkey for easier debugging.

More tips
Well, so far these were the problems I faced! :-) I'll update this as I keep discovering Greasemonkey's idiosyncrasies.

Sending emails via Gmail with Python

Whenever I need to send files from work to home I use gmail. Usually this email message consists of one single file (that may be a tarball) and has the file name for subject. In the "Automate Everything" spirit, I decided to build a script to do this task for me.

The first thing I had to do was find out how to send an email with an attachment via gmail. It wasn't too hard to find this information around the web, but it still took me the best part of an hour. So here's a simple Python script that sends an email with an attachment:

#!/usr/bin/python

import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email import Encoders
import os

gmail_user = "your_email@gmail.com"
gmail_pwd = "your_password"

def mail(to, subject, text, attach):
   msg = MIMEMultipart()

   msg['From'] = gmail_user
   msg['To'] = to
   msg['Subject'] = subject

   msg.attach(MIMEText(text))

   part = MIMEBase('application', 'octet-stream')
   part.set_payload(open(attach, 'rb').read())
   Encoders.encode_base64(part)
   part.add_header('Content-Disposition',
           'attachment; filename="%s"' % os.path.basename(attach))
   msg.attach(part)

   mailServer = smtplib.SMTP("smtp.gmail.com", 587)
   mailServer.ehlo()
   mailServer.starttls()
   mailServer.ehlo()
   mailServer.login(gmail_user, gmail_pwd)
   mailServer.sendmail(gmail_user, to, msg.as_string())
   # Should be mailServer.quit(), but that crashes...
   mailServer.close()

mail("some.person@some.address.com",
   "Hello from python!",
   "This is a email sent with python",
   "my_picture.jpg")

Back to basics: Vi

I've been doing a lot of programming in Ubuntu, mostly in PHP (my sister's in law web site), Perl and Python. I tried a lot of editors and (so called) IDEs, but was never happy with any of them. So I went back to basics: using Vim.

Now, I can ear the screams of Emacs users and the cries of all of you who only learned the :q! shortcut to exit Vi as soon as possible, but here's my view on things:

Emacs users: I tried using Emacs. I really did. And I actually used it for a long time while I was in college. But after using Vi for a while I had a really hard time both remembering the shortcuts and configuring Emacs to behave like I wanted to. So I just stuck with Vi.

:q! users: Vi is not that hard. Really! It does need a bit getting used to, but the freedom of not using the mouse and all those small shortcuts that you think you'll never use but turn out to be extremely useful make up for the learning curve.

To speed up your learning curve, here are the two secrets to loving Vi:

• A proper .vimrc file
• Knowing the right commands

Because there are so many "right commands" I'll leave that to a later post. Let's focus on the .vimrc file for now. And just to get your expectations in the right place, I'm not a Vim power user. If you look the web for .vimrc files you'll find files with hundreds of lines of code. Mine has 25, including comments... But it makes Vim behave like I want, so it's enough. Just go through the code and comments and you'll be able to understand what the configurations do. If not, drop me a line.

NOTE: The last 2 lines are wrong! Instead of [ you should have <. And instead of ] you should have >. But Blogger does a really bad job with these 2 characters....

" Turn on syntax highlight
syntax on
" Set tabs to 4 in size and make sure autoindent is on
set tabstop=4
set autoindent
" Make backspace work as in most editors
set backspace=indent,eol,start
" Black background and white foreground
highlight Normal guibg=black
highlight Normal guifg=white
" Make the default window size 50x80
set lines=50
set columns=80
" Store temp and swap files in this directory
set directory=/tmp/
" Show the status line
set ls=2
" Set the status line to:
" Filename [Modifed & Readonly Status] line number/total lines column (#buffer)
" More info at:
" http://www.winterdom.com/weblog/CommentView,guid,3372fa1e-966e-4389-b2ba-bb2ce527f52a.aspx
let &stl="%f %([%R%M]%)%=%l/%L %c (#%n)"
" Map Ctrl-Ins and Shift-Ins to clipboard copy paste (FIX THIS!!!)
map [C-Insert] "+y
map [S-Insert] "+gP

Objects in Perl? Use Python!

I've been meaning to try Python for a while. I'm kind of a sucker for programming languages, and knowing a bunch of languages allows me to choose the right tool for the right job. Turns out that this week I had the perfect opportunity to check out Python.

The task at hand was analyzing patterns in a bunch of XML files. I had 270Mb of XML files, and the largest file was 32Mb. Because I was pretty sure I was executing the analysis more than once (errare humanum est and I want to improve the analysis in the future) I decided to use SAX to read the files.

Having decided to use SAX I then decided to use Perl to do the job. I'm pretty familiar with it and was able to quickly find a SAX sample. Besides, I had a few string matches and replacements to do, and Perl is a great language for that.

Turns out that using SAX in Perl demands that you use and define objects in Perl. And it turns out that defining objects in Perl is... well, terrible! I really hated the syntax, bless and the way attributes were defined. It all looks like a big hack! To add insult to injury, the perlSAX has a few quirks when changing handlers. This is mandatory to make your SAX code maintainable... So I dropped Perl and went for Python.

To my surprise the transition was really easy. I was able to convert my Perl code to Python very quickly, with only a few doubts now and then on specific stuff. Here's what I gained from the transformation:

• I learned Python (finally!)
  
• Better SAX handlers (the quirks that happen in Perl don't happen in Python)
• Clearer attribute access (if you have an object with a reference to an array of references and want to print it in Perl... things can get weird)
• Clearer object definition and usage (no bless!)
  
• Fewer lines of code (from ~250 to ~150)
• Same performance (I was worried about this, but both scripts took the same time to execute!)
  

I still love Perl. If I want to parse a bunch of text files, do text transformations and the like it will be my 1st choice.

But whenever I need to do something a bit more complex that requires complex data types or OO programming, from now on I'll definitely turn to Python!

Web Testing

Last week I updated my sister's in law web site to have expanding folders. The solution I implemented is far from perfect but it does the trick. And since I used the prototype framework for the JS code, I figured it shouldn't have any problems in either IE or Firefox. I was wrong...

Turns out I forgot to test the web site in IE, and the first time my wife went there to take a look, it didn't work. Apparently nextSiblings doesn't work properly in IE... so instead of:

my_element.nextSiblings()

I had to do:

Element.nextSiblings(my_element)

This is really a pain for me because it means I have to test the site in both IE and Firefox all the time. And since I use Ubuntu, that didn't seem like an easy task. Wrong again!

There's a project called IEs4Linux that allows you to run IE on any computer that runs Wine. So I followed the instructions on the website, installed the thing, and in about 5 minutes (mostly because of download times) I was running IE 6 on my Ubuntu machine. I actually found out and solved the above problem all in Ubuntu. Sweet!

Use source control. Always!

So you're doing this simple project, you have a bit of PHP, a bit of mySQL, you add a dash of really cool JavaScript, you publish it to the server and... BANG! Everything stops working!

Guess what? It happened to me this week. Because of a rather simple thing, actually. In my development machine I have mySQL 5, and the server machine has mySQL 4. My beautiful queries were all failing.

It actually took me about 2 minutes to fix the problem. Get the latest running version from source control (I'm currently using SVN) publish it and that's it. And during those 2 minutes I just kept thinking, "I'm *so* glad I have this in source control!"

Listen to experience. Use source control. Always!

Automate Everything!

The other day, while waiting for another computer reboot or something similar, I grabbed The Pragmatic Programmer from the shelf. I already read the book, so I was just quickly browsing the tips in the book when I found this one: "Automate Everything". (you can find it by looking for "Don't Use Manual Procedures" in the previous link).

I already do this at work, but I started thinking it would be a really good idea to start applying this principle to my home projects. Take my sister's in law website. Here's the procedure I use to upload it to the server:

1. Open googleanalytics.txt and copy the contents
2. Open pageend.php and paste googleanalytics.txt content
3. Open gFTP
4. Open KeePassX to fetch the site's password
5. Use gFTP to transfer some of the directories in the development folder (this actually counts as about 10 steps!)
   
6. Check if I didn't make a mistake in the gFTP 10 steps!
   
7. Open pageend.php and remove googleanalytics.txt content

Some notes on these procedures: I never did steps 1, 2 and 7. It was too much work so I never actually got Google Analytics to work until yesterday. I'm not kidding about step 5 being about 10 steps... I actually uploaded the folders one by one to make sure I got all the right files and that the .svn folders weren't copied. This may seem rather stupid, but since I executed this procedure about once a month it wasn't really a problem, just a nuisance.

Still I decided to automate everything. I did a shell script and right now I have a one step procedure to upload the site:

1. Run ./publish.sh

And what did I gain with this 2 hour or so investment?

1. I learned more about bash programming
2. I learned how sed works
3. I learned about wput
4. I got Google Analytics working on the website
5. I'll be doing no more mistakes while uploading the site
6. I can make small changes to the site without thinking about the dreaded upload process
7. I feel like a better geek! :)

3 Cheers for Open Source!

I'm a great user and adept of free software, but I must confess that I'm more attracted to the "free beer" side of things rather than the "free speech".

But for some reason each time I change a bit of open source code and make things work just like I want them to, it gives me an enormous satisfaction! There's 3 episodes that I really enjoyed on my open source adventures:

1. I patched the linux kernel! It was just a small patch to make my CD-Rom drive work, but it was tremendously rewarding to see the little light blink on that drive! (I didn't publish this one because it was a really old cartridge 1x CD-Rom drive)

2. I patched Kannel. This was amazing because I did the patch at the same time someone asked for it on the mailing list. So I send it to the mailing list and it is now part of the Kannel code!

3. I've been trying out Total Commander on Windows. It's a really good file manager and has an iPod Plugin. Turns out the plugin doesn't work with newer iPods. But since the source is available at the author's site, I downloaded it and fixed the problem! (I'm now talking with the author to get the patch published)

The funny thing is I believe I would've done the same even if the software wasn't free as in "free beer". Actually I would be thrilled to be able to fix some bugs and annoyances in some paid applications. Unfortunately I don't have access to the source code, so I'll just have to live with them...

OutSystems Express Edition

If you're looking for a fast way to build an enterprise application, if you want to be able to change it even faster and if you want it for free, OutSystems Express Edition is the answer!

I've got to be honest here. I work for OutSystems so you may consider my opinion to be a little biased. But the great thing is that by subscribing to the beta program you can see for yourself how fast it is to create a web application using the OutSystems Platform.

So go ahead! Register now for the beta program, wait impatiently for the "Welcome to the Beta program email", take a look at the bundled IT Asset Manager and build your own web application!

Don't forget to send us all your feedback about the OutSystems Express Edition so we can improve the product! And feel free to exchange ideas with other OutSystems users at the OutSystems Community!

Happy developing!