Hacking Lua (in Xcode) 1

Posted by Nicholas Chen Sun, 15 Feb 2009 21:03:00 GMT

I have always been fascinated by application virtual machines. And I have been trying to find a simple but well-documented one to work on for fun. There is a proliferation of Javascript VMs out there but they are pretty big (V8 is the latest kid on the block and it's about 100 000 lines of C++, not my favorite language) and aren't as well documented.

There are some smaller ones like tinyrb and tinypy but the goal of such projects aren't really clear. I am not sure how much of Ruby/Python they are trying to support. They are great for hacking on if you have some experience with VMs already and can contribute to the design/implementation. But otherwise, it can be pretty hard to comprehend what is going on.

Both tinyrb and tinypy did mention that they were trying to emulate the Lua VM. So I decided to check it out. The Lua VM (like the Parrot VM) is a register-based VM, which is pretty unusual since most VMs are stack-based. Its use of registers instead of a stack makes it more fascinating to study.

And I like what I have seen so far. There is good documentation on multiple levels: the language itself, the architecture of the VM and plenty of examples. Also, the code is in C which is less messy than C++. The Makefile is simple and isn't using any cryptic tricks. And Lua itself seems like a simple language to learn.

So my plan is to dive into the source for Lua and then see what can be done to improve tinyrb. But first, I need to set up an environment to develop in.

I could certainly continue to use vim/TextMate, but I have been using Xcode more because I am doing some iPhone development and have gotten use to some of its features. And I like its built-in debugger which saves me from having to interact with gdb directly for the common tasks. In fact, I am not sure if there are better C/C++ IDEs for OS X. The Eclipse CDT project is the closest that I can think of but I haven't used it much.

So here are my steps for doing this: from getting the source into Xcode and also starting a debugging session. I am using the All-in-one look for Xcode so your UI might look different.

Grab the source for Lua-5.1.4 from http://www.lua.org/ftp/lua-5.1.4.tar.gz. Unarchive it. For this article, I will be unarchiving it in ~/Xcode.

Create an External Build System project

Fire up Xcode and create a new External Build System project. I called my project Lua. You can put the project wherever you want. It doesn't need to be in the same directory where you unarchived the Lua source code since we will be referencing the source files anyway.

Add the existing Lua files

Right-click on the project and add the entire directory where you unarchived the Lua source code. In my case it would be ~/Xcode/lua-5.1.4.

After adding the files

This is what it looks like after adding all the files.

Make changes to the Makefile for OS X

Next, we need to edit two of the Makefiles. First, we need to edit the top level Makefile in ~/Xcode/lua-5.1.4. Change line 8 to PLAT= macosx since we are targeting the OS X platform. Then, we need to make changes to the Makefile in ~/Xcode/lua-5.1.4/src. Change line 17 to MYCFLAGS+= (we want the + symbol). Change line 108 to $(MAKE) all MYCFLAGS+=-DLUA_USE_LINUX MYLIBS="-lreadline" (again, we want the + symbol). In fact, by default, it should have the + symbol so that we can easily accumulate our own settings. I suspect that this is a careless omission in the Makefile for Lua.

Changing the parameters of the Lua target

We need to make some changes to the Target configuration for our Project. So navigate to the Lua target in your Groups & Files (it's the left navigation pane). And double-click the Lua target. Then edit it so that it resembles the image above. Notice the line that says "Settings show Debug" near the top of the window; we are changing the settings for Debug (and not Release).

You need to change the directory to the location where you unarchived the Lua source so that it can find the top-level Makefile.

Then you need to add a new build setting, MYCFLAGS so that we can pass the -g flag (for debug) to the compiler. Without this flag, we cannot debug our program effectively with the debugger (it doesn't have all the necessary debugging information).

The result of the build (for debug)

Now we can build our project by going to Build > Build. And notice that we are building for the Debug configuration i.e. on the menu item at the top it says 10.5 | Debug | i386. The settings that we made for our Lua target are reflected in the output of the build e.g. gcc -O2 -Wall -g .... (the -g flag that we specified in the settings).

Executing Lua in the command line

Now that the project builds, we can proceed to debug it (or rather we just want to step through its execution in the debugger to find out how it works). The image above shows how we would run that executable (notice that we are passing some arguments to it). We want mimic this and run the program in the debugger.

Create a new executable that you can launch from Xcode

So we create a new Custom Executable. We call it Lua.

Parameters for Lua custom executable

In the dialog that pops-up, remember to specify the location of the lua executable (you will only have it after you have built the project).

Don't load symbols lazily

Before we can debug the lua executable, there is one more setting that we have to perform: change the way debugging symbols are loaded. So, open up the preferences for Xcode, and switch to the debugging tab. Ensure that the "Load symbols lazily" is unchecked.

Build and Debug your program

Now we need to set a breakpoint. Notice in the image above that I have set one on line 378 in the lua.c (Xcode/lua-5.1.4/src/lua.c) file. After that we can invoke the debugger from the Build > Build and Debug menu.

Debugger in action

And finally we have our debugger in action.

The steps above are applicable for any other external project that you need to work on (sans the modifications to the Makefiles since that is Lua-specific).

Online Gradebooks - Something Worth Hating

Posted by Nicholas Chen Sat, 20 Dec 2008 01:18:30 GMT

Instructor Help: Reorder Grade Book Columns:

"The following tutorial shows how to reorder and hide/show columns in the Grade Book. Each Grade Book view tab (Grades, Members, View All, Custom View, SCORM Grades) has its own independent display order. The order for a specific view tab can be changed by pressing the Reorder Columns button when viewing that tab. If you wish all of the views to display in the same order, use the Reorder Columns button for each view."

The University of Illinois uses the same grading system that the University of Idaho does. And it is fortunate too since I was having a hard time trying to figure out who to reorder some of the columns. It is so hard that I actually need to search for instructions on how to do it from the internet. It is always a bad sign when something as simple as reordering some columns requires the user to read a manual.

I had optimistically assumed that in this modern age of AJAX that a mundane task like reordering columns was basically solved. I was proven wrong.

Before you read more, it might be best to read the instructions above first to see how unintuitive the UI is for the Blackboard Learning System really is.

Compare this to how a simple (and useful app) like Basecamp does it:

Basecamp help: To-do lists:

"To the left of each title, you'll see the reorder icon (it looks like four arrows). Click the icon and drag the list up or down to the location you want."

Simple: drag-and-drop.

And to make matters even worse, the entire Blackboard system is written in Java, takes a long time to load and does not even take advantage of all that Java has to offer. I am sure someone can come up with a better UI in Java. How hard can it be to support drag-and-drop in Java?

No one (student or instructor) that I have talked to likes using this online gradebook. So it really makes me wonder, is this the best that software developers can do for online gradebooks? Or is it this bad just because there aren't enough competitors in this field?

Who The Hell is Bob Costas?

Posted by Nicholas Chen Sat, 09 Aug 2008 02:47:00 GMT

At the recommendation of my mom back home, I decided to watch the opening ceremony of the 2008 Summer Olympics. Disappointingly, I did not have right cable package so I could not watch it live with them.

Instead, I had to wait until 7:30 pm (it was happening live at around 7:00 am local time in Beijing) to watch the delayed telecast on NBC. I was actually looking forward to it based on what I had heard and read from the Internet earlier today.

The telecast started off well with the spectacular performances in the Bird's Nest stadium. I was really impressed by what the Chinese had put on. By this point, the commentator had already begun to annoy me a bit with his weird, uncalled-for and out-of-place comments. But there was nothing really bad about his comments yet.

Then the time came for all the contingents to come marching in. And here is where Mr. Bob Costas started making all sorts of weird comments. I cannot remember all of them, so I have to paraphrase it.

"Of the xxx number of countries participating here, most of them will not receive a medal".
"Malaysians are rather particular about people knowing where their country is. They get upset when people don't. So let's see where the country is...."
"The president of xxx promised that the athlete who brings home a medal will be awarded a dream house... looks like he has nothing to worry about since there probably wouldn't be one."
"This nation's flag bearer was chosen through a series of text messages. She had 73% of the votes. I bet she was sitting there texting in her name all day..."
"And here are the xxx who have won more medals at the Math Olympiad..."
... And many others that I can't remember. I am sure someone else will blog about this soon.

I don't think I am overly sensitive to his comments. I can definitely take a joke or too, but the inanity of his comments just puts me in an unhappy mood.

It seems that this is not the first time he has done something like this. His wikipedia entry tells of his past faux pass (if one can still call them that; they seem more like verbal crimes):

"During the 1992 Barcelona and 1996 Atlanta Opening Ceremonies, Costas' remarks on the China Team's possible drug use caused an uproar among the American Chinese and International communities. Thousands of dollars were raised to purchase ads in the Washington Post and Sunday New York Times, featuring an image of the head of a statue of Apollo and read: 'Costas Poisoned Olympic Spirit, Public Protests NBC'."

I will be looking forward to the closing ceremony but hopefully not with Bob Costas. I am also rather surprised with NBC's decision to put Bob Costas as one of the commentators given his less-than-impressive history with making such blunders.

A Pattern Language for Screencasting

Posted by Nicholas Chen Tue, 03 Jun 2008 02:24:02 GMT

About 9 months ago, I said that I will be looking into collecting different patterns for screencasting. So finally....

I just finished writing a pattern language for screencasting based on my experiences producing and observing various screencasts on the internet. Through my experiences I have noticed various patterns that myself and other screencasters have used. Applying those patterns would certainly help the presentation of your screencast but, more importantly, I believe that the patterns enhance the teaching-learning experience of screencasts. And ultimately that is what matters most.

Interested readers may view the patterns here. The pictures that I have included are based on software for the Mac since that is what I use for my screencasting.

At the moment, it's a rather complete version of what I had originally envisioned for such a document. It can definitely be improved but I would like to get a version out on the web for people to read in case anyone is interested in providing feedback or suggestions.

I do maintain copyright on the document in case I ever decide to publish it at some pattern conference.

Antony Blakey's Efforts on VisualWorks

Posted by Nicholas Chen Fri, 30 May 2008 23:10:00 GMT

The end of LLVM/WebKit VW integration and layout improvements.:

"Over the last 18 months or so I've put a lot of work into enhancing VisualWorks with WebKit integration, GCCXML, and later LLVM integration, as well as improving the GUI layout system. I've stopped development on these projects, primarily because there seems to be no external interest in these projects."

I am really sad that Antony Blakey is going to stop work on the GUI layout improvements for VisualWorks. His work actually made the VisualWorks UI pleasant and usable. Here's a screenshot to illustrate what I mean.

VisualWorks 7.6 on Mac
On the left is the version enhanced with Antony's packages. And on the right is the original UI look of VisualWorks for the Mac. Notice how the tabs, find menu bar and the fonts look so out-of-place for the version on the right. And notice how they look more native for the version on the left. If you just glanced at this you might not find them to be much different but if you had to use the layout everyday, which would you actually prefer?

I am glad that he has pushed his GUI layout work onto the Cincom public repository so we could actually look at it and modify them as necessary. A cursory glance reviews that all of the code is written in Smalltalk without the need for calling C code. So there is the possibility that I can actually decipher the code and make changes if necessary. I am not an expert on VisualWorks so this might still prove to be rather challenging.

On the other hand, while I am definitely impressed by his work on LLVM/WebKit integration, I am not sure if I have the right level of knowledge to hop on board and contribute. I definitely would like to and perhaps after a year of using VisualWorks I would be able to. But right now, I wouldn't be of much help. Moreover, if I read his blog entries on these subjects correctly, the LLVM/WebKit integration involves diving into the VM source code and actually changing the internals of the system. I think it even requires a custom version of the VM. Also, he hasn't pushed all of his work on LLVM/WebKit integration onto the store (he has an interesting article on why the Cincom public store is not really suitable for collaborative development)

Most people might argue that the VisualWorks UI is actually much better than the one for Squeak with all its colorful windows and non-standard layout. But here's a quote that exemplifies what UI design (taken literally as a quote like I would from Shakespeare with none of all the mumbo jumbo of religion):

"So, because you are lukewarm – neither hot nor cold – I am about to spit you out of my mouth. – Revelation 3:16"

So the Squeak UI isn't all that bad. It's different enough that your mind just accepts it as being different. On the other hand, the standard VisualWorks UI for OS X tries to blend in but does it in a way that is just lukewarm... This isn't to say that the Squeak UI or the VisualWorks UI can't be improved on....

In short, I am really impressed by Antony Blakey's efforts. He definitely has some radical ideas for improving (modernizing?) Smalltalk, hence, the title of his blog: "The creation of a new order of things". I will definitely continue to monitor his future work on Smalltalk.

Drag and Drop on Mail Dock Icon 1

Posted by Nicholas Chen Fri, 30 May 2008 17:29:00 GMT

Programming Quotes, Sayings about Programmers, Software Developers, Application Development:

"The only way for errors to occur in a program is by being put there by the author.  No other mechanisms are known.  Programs can't acquire bugs by sitting around with other buggy programs.  ~Harlan Mills"

The last time I updated to OS X 10.5.2, some developer inserted this extremely annoying bug into Mail-- dragging any file onto its dock icon would crash it! This is particularly annoying because I already had it wired onto my reflexes to perform a drag and drop every time I want to send an attachment to someone.

Eventually I was able to find the solution: change Mail's composing preference to RTF instead of plain text. Well that isn't really a solution since that means that I will have to compose all my e-mails as RTF which is something that I don't agree on. E-mail messages should be plain text whenever possible! RTF might be better than HTML but it is still egregious compared to plain text.

Anyway, I found an alternative solution. Instead of dragging the file to the dock icon, create a new empty message first and then drag the file onto the body of the message. This involves the extra step of creating a new empty message but at least it does not crash Mail nor force me to use RTF. Being able to drag the attachment into the mail message body without crashing it (even with the plain text setting) just illustrates that this is a bug that could have been avoided/fixed since there is nothing inherently wrong underneath with Mail. Instead it's probably just the drag and drop component that needs some reworking.

Yesterday I upgraded to OS X 10.5.3 because I wanted to install the latest iPhone Beta SDK. The README specifically said that OS X 10.5.3 is required.

And I was rather disappointed (but not really surprised) to discover that this Mail bug has not been fixed yet. Fortunately I could use the alternative solution which will suffice until this bug gets fixed (if ever). I'd rather have less features (for instance remove the useless Stationary/Template feature from Mail) than to have a program crashed on the most rudimentary of tasks.

(Finally) Some Good Use for the Camera Phone

Posted by Nicholas Chen Sun, 06 Apr 2008 00:56:00 GMT

Evernote

I've been reading some mentions/reviews of Evernote on various sites so I decided to try it out for myself. Right now, the latest beta version is by-invitation only. I tried my luck at their invitation-only beta registration and was lucky enough to get an invite the next day. Since then I have sent out 5 (of the 10 invites) to my friends.

The coolest feature of Evernote is its ability to recognize text within images (see this demo video). The ability to recognize text in images is not something new but this is the first time I have seen it offered in a consumer application. Because of this, I finally found a good use for the built-in camera of my iPhone.

Text recognition is not built-in to the application itself. Instead anything that you store in Evernote is synchronized with the Evernote's servers (this allows you access your notes from the web). The text detection/processing is done at the servers and the metadata is sent back to your application. That means that unless you sync with the servers, you can't actually get text recognition. I think this is a good idea since it lets Evernote change its recognition algorithm without bothering the user with some software update.

So how good is the text recognition? I decided to do some experiments and come up with the following observations. These are merely observations so they are not rigorously tested. Also, expect the recognition capabilities to increase over time.

Old Intel Logo
It recognizes some non-standard letters like Intel's old logo on this mouse pad.

Vertical Intel Logo
It does not recognize vertical text well.

Is there an 'i' there?
In fact, for vertical text, its recognition seems to be pretty off; recognizing things that are not there.

DTrace Event Poster in Evernote
It probably only recognizes words in the English dictionary since it can't recognize "DTrace".

DTrace Event Poster in Evernote
It recognizes most words but might have some trouble pinpointing their exact location.

Symbol recognition in Evernote
I don't think it recognizes symbols well; it's probably optimized for numbers and letters.

Multicolored Word in Evernote
It seems to have trouble with multi-colored words like the Google logo.

So for better results, the text should be horizontally written, unicolored and contain only alphabets and numbers. But it works pretty well for most cases. It certainly beats having to enter information by hand.

So, will I continue using Evernote after its beta period? I am expecting that it will require a paid subscription for the ability to sync and recognize text within images but if the price is reasonable, I might continue using it. Moreover, if they decide to create a native iPhone app (an offer it for free for paid subscribers), the likelihood of using it is even higher for me. After all, there isn't a convenient way to sync notes between my iPhone and my computer. I'll definitely keep Yojimbo around though since it handles PDF and has some nice predefined containers for storing image, serial numbers, etc.

External Monitor Setup 1

Posted by Nicholas Chen Fri, 21 Mar 2008 20:42:00 GMT

After spending about a year with two huge and bulky CRT's in my office, I decided to spend about $700 to get a new 24 inch widescreen display from DELL. It wasn't only the physical size of the CRT but also its blurriness that annoyed me. Here's a picture to show what my desk looked like before:

Old Setup
Both CRTs are Dell M993s. Surprisingly those monitors could display 1600x1200 pixels! However, that comes at the expense of blurriness and eye-ache after a couple of hours.

Here's a picture to show what my desk looks like with the new monitor:

New Setup
The Dell 2408WFP monitor connects to my MacBook Pro. The other smaller monitor connects to my Windows machine. I share the mouse and keyboard using Synergy.
The smaller monitor is borrowed from my colleague's table since he won't be around until Fall 2008.

Using a large monitor is not without its challenges and problems. The main problem on the Mac (and possibly Windows and Linux) is that it becomes rather troublesome to resize the windows. Here's why: all windows manager that I know have three buttons on each window: close, maximize(expand) and minimize. On a small monitor, clicking the maximize button expands the window to fill your small screen -- usually a desired behavior so you can see more of the contents in the window. On a large monitor, clicking the maximize button, expands the window to fill your large screen -- usually an undesired behavior since you are wasting screen space.

Unfortunately, I have not found a nice way to resize the windows yet. I do have a trick though. Since my external display is powered by a MacBook Pro, I can always move the window over to the MacBook Pro's smaller screen, maximize it and then move it back to the external display! It's cumbersome but it works. And I get uniformly sized windows -- I am rather particular about having my windows be about the same size.

Fortunately things are better with my browser since I can just rely on javascript bookmarklets:

Pimp My Safari: bookmarklets:

"Resize to 1024x768"
javascript:void(window.resizeTo(1024,768))

Updated to Typo5

Posted by Nicholas Chen Fri, 21 Mar 2008 00:37:00 GMT

So I discovered that my web host just upgraded to Rails 2. How did I make the discovery? Well, both my blogs were malfunctioning. And when a rails web application malfunctions, it is usually a good idea to check if they had upgraded it. Previous experience with Typo should have taught me that but sometimes I forget such things.

I should definitely freeze the rails version right now so that I don't encounter such problems in the future.

Anyway, I grabbed the latest version of Typo and installed it. I must say the latest version really looks slick. I chose to use the same Scribbish theme for this blog but there are two new themes that are pretty nice too.

One caveat is that some of the plug-ins have been removed from the base Typo installation. This caused me some problems as I migrated my blog over. I could have just installed the old plug-ins following the instructions from here but I decided to stay fresh. I am not sure whether there is an easier way to do this, but I basically went into the database and remove the sidebars table:
DELETE from sidebars;

I am going to upgrade the softwareengineering.vazexqi.com blog in the next few hours so it should be back up by tonight.

vazexqi.com updated

Posted by Nicholas Chen Mon, 17 Mar 2008 04:36:35 GMT

vazexqi.com updated!

After delaying for a very long time, I have finally updated the root vazexqi.com site and turned it into something that reflects more of what I am doing as a Ph.D. student. I was only compelled to do it because the CS department demanded a curriculum vitae from each of its students. I wanted to put a web address on my CV so that people can keep up with what I am doing and find. However, I did not feel that it was appropriate to put my blog address directly on it; after all, a website that starts with the word "blog" might not sound professional enough to some academic peers (it should be fine for a developer's blog though).

I was too lazy to actually write everything from scratch so I decided to give RapidWeaver a try. I had used it before for various tasks but I am only needed to use the demo version since I would just borrow (and credit) their templates and then code everything up by hand. But this time, I decided that since vazexqi.com was going to be mostly static content, I needed a convenient way to easily update it using some application. Also, I am not really a web designer so I would like to be able to quickly change the template without having to do too much work on my part.

Overall, I think the new look is pretty decent. There are things that still need updating but overall it's a good start. There are two things now that I must remember to do: constantly update the static pages and upload a picture of myself to the front page....

I don't really subscribe to the idea of putting yourself out there on the web for everyone (like this guy) but I do think that it is important to have a clean web page where your peers can easily find you.