This Old Code Still Compiles

Message to my younger self, in case you time travel and find this blog:  You won't realize it while you're working on that project, but you're learning life lessons about software development that are going to apply to dozens of future projects, so pay attention. 

Recently I found my almost-two-decade-old C code from the first "big" software project that I created (about 2000 lines of code in 20 files). It was the first "real" product I made that actually got used. Last night, after I installed the termcap and ncurses libraries on my Linux box, the code compiled with no source or Makefile changes (although there were a lot more compiler warnings than I remember!).

Looking back there are at least two things I learned on that project back then that are timeless.  Of course I didn't know it then, but it was early training for real-world projects.

1. You'll often need to learn something totally new just to get the job done.
2. Users really like output. And they like it a certain way.

Background: the Timeclock Project

The project was a terminal-based timeclock program for the students who worked in the college computer labs. The 25 student T.A.'s who worked the lab would work varying length shifts and this program handled clocking in/out, saving the data, and it generated timesheets for the computer lab manager. It didn't have a ton of features, but I was pretty proud of it.  

Frequent Compilation

I used a basic terminal editor (vi) to create it; thus there was no syntax highlighting and no red squiggly lines to point out syntax errors. My approach was to execute a 'make' from within vi after writing only a few lines of code. Next time your fancy IDE lets you know about syntax or API mistakes, stop for a second and try to remember what it was like before these features.

A Library For the UI

Does anyone here remember curses? That C library made it possible to create a terminal program that performed basic text UI functions. Until this project I had only done basic print-to-screen and read interaction. So just to make a decent interface I had to learn this library. Have I used this library since then? Not until tonight when I installed it on my Linux box to make this compile.

Specific Output

Next, the college's business office had a specific process for handling student timesheets. Paper timesheets of course. The timesheet form they used had been copied and re-copied until it was a blurry but still mostly readable form. How was I going to make output look like that? Well, you may or may not believe this, but sometimes people produced output before PDF existed. We had laser printers - this wasn't the dark ages after all. So I went to Barnes & Noble and bought a book on how to create a PostScript file and over the next two days I learned PostScript and make a form that looked just like the college business office's timesheet form. It was such a good-looking form that the business office started using that as the master copy. A couple years later when I was back on campus I noticed they were still using it, except by then it was a photocopy of a copy of a copy...

A Couple Short Semesters of Use

The timeclock software didn't last longer than a few short semesters and it never expanded to other departments beyond the computer lab TA's. There were a few requested changes to it along the way (user management, supervisor editing for tardy employees, etc).  I had high hopes for its continued use, but alas... I had graduated and they eventually stopped using it.  

Those Lessons I Mentioned Earlier 

1. Learn Something Totally New to Get the Job Done
Every new project could necessitate a new technology or library that you've never tried before. (For example, I've been using Java for a long time and I've never used it to communicate to hardware through a USB port.) I'm not saying you should go out of your way to try to use something new on every project; it will just happen more often than you might expect. Approach this carefully, analyze your options, and get some recommendations from others if it's a totally new technology for you.

2. Users Like Output
Like it or not your users often probably don't want to use your program. Your users want their output and your program is (hopefully) the most effective means for them to accomplish that goal. Depending on the software and the user-base, (specifically I'm talking about PDF or report-generating software), there are going to be times when the output looks perfect to you. Then a user will email you and want to know why a word is in the wrong spot or something doesn't line up just right. Listen to your users (and especially listen to your testing group). Take pride in making your output perfect. If you're lucky it will be

Rooting an old Samsung Intercept

A friend gave me her old Samsung Intercept though she didn't know why I'd want her "crappy old phone."  And I must say, with all the bloatware that Sprint has loaded up on this seriously underpowered phone, it was indeed a pretty crappy experience.

From what I've read online, there were a lot of complaints about the Intercept, but I'd place most of the blame for its poor performance squarely on Sprint.  Before I started messing with it I should have taken an inventory of all the services that Sprint had running in the background. There were a lot.

Anyway, this was my first attempt rooting a phone and putting on a custom ROM.  I'm still new at this and I figured I should take a few notes before I forget everything I've done to it.

Disclaimer: Blah, blah, blah, you might destroy your phone; don't blame me if you do.

Overview:

1. Install custom "Recovery" tool  (CM01 Custom Recovery)
2. Install custom ROM (essentially the OS the phone runs)

I think of the "Recovery" mode on a phone as akin to the BIOS on a PC. Depending on the phone, you hold down a few different buttons and press the power button to enter Recovery mode.  For the Intercept, hold the Volume-Down and Talk buttons when you power on (this usually takes me a few tries to get the timing right. If the progress bar goes past the Sprint logo at the bottom it didn't work).  The reason we need to replace the Recovery mode is that the one preinstalled on the phone does some kind of check before installing a ROM update to make sure it's "approved", i.e. that it comes from Sprint.  So we replace it with a better one, in this case CM01 Custom Recovery, which originated with the CyanogenMod folks.

There is a wealth of information on the SDX-Developers forums.  The guide I used was one titled "[GUIDE] - Install CM01 Custom Recovery using SWUpgrade Tool."  The steps involved installing a program "SWUpgrade" onto a Windows PC and downloading a few other things. I'm not going to repeat it all here but be careful that you don't try to install the Samsung Moment's software on top of your Samsung Intercept. That won't be good for anyone.  There are other methods of getting root and then installing the CM01 Recovery, but that's what I used.  (Another thread says you can just download an apk from somewhere and gain root that way.  I didn't try it.)

So anyway, that gets the CM01 Custom Recovery installed, and from there I was able to install and try a few different ROM's, with varying degrees of success.  (Note: if you see references to DL05 or EC07, that's apparently two different kinds of Intercept: DL05=Sprint, while EC07=VirginMobile).  I just copied the appropriate zip file to the SD Card in the phone, selected it from the Recovery menu, and installed. (Sometimes doing a wipe beforehand to make sure the phone is fresh).

ROMs I've tried so far
(mostly found from another SDK Developers thread on ROMs for the Intercept):

• inxane0381intercept - this didn't work for me at all;  the screen was dark and the phone just sat there and buzzed/vibrated in a dot, dit-dit-dit pattern. Kinda scary and thought I'd bricked the phone, but fortunately got back to the beginning. 
• 
  Stock ROM on Crack for DL05 - works great. phone is way more responsive now
• Ice Cream Froyo - looks like ICS but still Froyo, partially didn't work for me.
• Back to the original FB01 Stock ROM.
• Ubuntdroid 6.0 DL05 - which worked okay and had neat graphics but didn't seem particularly fast and apparently has nothing to do with Ubuntu, they just like the name and logo.
• Another thread says there's a working ROM of CM6 for Intercept available on 4shared - I'm trying this one now, my fallback will be to use the Stock ROM on Crack since that seems to be the best thus far.

Summary:  While this was fun and all, I really don't feel like I did anything special or creative -- I just followed some existing guides, following work done by those who came before me.   I think a better challenge would be to build Android from source and see how that goes.

You're Not the First to Have This Problem

This is going to sound like the "compiler bug" problem, but this time it was something very similar. Yes, there are probably some bugs in the JDK. The likelihood of there being a bug in any software increases along with the size of the codebase. (Unsubstantiated claim, but I provide no footnotes with this blog.)

Anyway, when you hit a problem keep in mind that you're never the only person to have encountered this problem. Somebody else has seen it and probably been in a more desperate situation to need a solution.

Every once in a while I'd see some odd stuff going on where a bunch of threads seemed to get stuck. Those threads interacted via a CORBA interface, so I just chalked it up to some CORBA-wackiness and thought I'd try to code a way to kill off those threads when they got stuck waiting for CORBA to finish.

Sidenote:  CORBA came from the OMG, which has the most ironically appropriate acronym ever invented, because "it's like, OMG, could there be anything better than a consortium named OMG!?"   By the way, do not click over to the talk page on Wikipedia entry for OMG if you're in a quiet professional environment -- I started laughing so much my wife thought there was something wrong with me.

So anyway, I got a recent thread dump (not a stacktrace, since those threads didn't throw any exceptions, they were just waiting) and decided to look around a bit for the answer. Here's a sample of one of the stuck CORBA threads:

"pool-538-thread-1" - Thread t@67441

   java.lang.Thread.State: WAITING

        at java.lang.Object.wait(Native Method)

        - waiting on <5dfd006a> (a java.lang.Object)

        at java.lang.Object.wait(Object.java:485)

        at com.sun.corba.se.impl.transport.CorbaResponseWaitingRoomImpl.waitForResponse(CorbaResponseWaitingRoomImpl.java:140)

        at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.waitForResponse(SocketOrChannelConnectionImpl.java:1084)

        at com.sun.corba.se.impl.protocol.CorbaMessageMediatorImpl.waitForResponse(CorbaMessageMediatorImpl.java:253)

        at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.marshalingComplete1(CorbaClientRequestDispatcherImpl.java:362)

        at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.marshalingComplete(CorbaClientRequestDispatcherImpl.java:336)

        at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.invoke(CorbaClientDelegateImpl.java:129)

        at com.sun.corba.se.impl.corba.RequestImpl.doInvocation(RequestImpl.java:309)

        at com.sun.corba.se.impl.corba.RequestImpl.invoke(RequestImpl.java:230)

        - locked <6dae8354> (a com.sun.corba.se.impl.corba.RequestImpl)

        [etc., etc...]

I stumbled around the Interwebs for a bit and happened upon the inevitable similar stack traces and mentions in a forum or two or three, but none of them had the answer. Then I expanded my search a bit and  found an existing item (7016182) in the Java bug database... submitted about a year ago and a status of "Cause Known" but now what?  So I searched on the related bug ID numbers and found yet more info in number 7046238. Yay!  Finally an answer!  It says it's been fixed in JDK release 6u27 which I'm pretty sure is 1 update newer than what we were running.

That's good news, but it still needs to get tested. In the meantime, I compared the source of the affected JDK class CorbaClientRequestDispatcherImpl with the the fixed version. Yep, just as the bugfix note said, the fix is a change to how the synchronization is handled and it makes me glad I didn't have to try to figure out how to fix it. In a way it's comforting to know that even the experts sometimes make mistakes.

Anyway, that was my adventure this weekend. What did you do?

Modeling Intangible Many-to-Many Relationships

Modeling intangible things in a data model takes practice.  Here's an example I've used and choices made along the way.

Visible and Real

Set aside the database "join table" for a moment and consider how you represent your primary entities in code.  Sometimes it's easy to see how the concepts should map to database tables and Java classes -- especially when the concepts are relatively visible in the real world.

Let's take, for example, my favorite model:  Creature - Skill - Achievement.

Creature is obviously a visible "thing" that exists in the real world.  Skill and Achievement aren't quite as visible except in a temporal sense, but at least while you're watching a Creature swim (or whatever other activity), you can see that it's happening with some level of skill.  An achievement like "first place" or "swam 10 laps" may be represented by an award of some kind but there the visible thing is just a reminder of an achievement, it's not actually the achievement itself, but somebody saw it happen so at least for a moment it was real.  So really this one isn't so hard to wrap our minds around.  We can have Java classes with a 1-to-1 correlation with the database tables and it all pretty much makes sense.

Invisible Things

But when entities represent something intangible the mapping isn't as clear.  A few years back this situation cropped up in a development effort where I was a (pseudo) dev lead.  Our application was going to deal with People and Decisions.  In this instance a "Decision" could be something like a "Decision to Exercise Regularly."  Each Person could have many Decisions.  Each Decision could be made by several People.  Thus, a many-to-many relationship.  So how do we model this?

or translated into something nearly equivalent...

The database folks already had the tables set up as  Person, Decision, and PersonDecision which followed their standard naming conventions.  And, at least with a traditional database, this is how a many-to-many relationship is created.

Option 1.
Should we stick with a 1-to-1 mapping from database table to Java class?  In that case a Person object would have a collection of PersonDecision objects and some field there in the PersonDecision class would show what kind of Decision was made by the Person.  This option has the benefits of being a simpler implementation for mapping.

Option 2.
However, it could be argued that each Person object should directly have a collection of Decision objects, with any necessary metadata from the database's PersonDecision table as fields in the Decision class.  This option has more object-oriented beauty about it and avoids the awkward PersonDecision name.

Consider the Usages (and the kittens)

We could have talked about it for days but there was a lot of work ahead of us so I went with my gut and chose for us to go with the first option (though there was some further discussion expressing respectful disagreement).  The major factor for me is that there was additional data attached to both Decision and PersonDecision (such as EventDate).  Also, there were some usages within the application that involved only Decision and not Person or PersonDecision.  So even if we had merged the data from the two tables into one class, it would still have required a Template Decision  class representing an archetype of a Decision separate from a Person.  Thus we would have still needed an additional class, just perhaps with a different name, putting us back into situation we started with plus some additional complexity.

Who really knows whether or not my choice was the correct one.  It's the one we went with on that project and it didn't feel like we had to jump through a lot of hoops to make things work.  Revisiting it, maybe it would be nice to implement it both ways to compare the merits of each option more closely.  But that's a topic for a different post.

Build Your Own Facebook - Part 1

Here's a billion mythical dollars - let's design the next popular mobile/social application.  I don't know how long this series (!) of posts will last but I'll at least start with this one.  We're not actually going to build a Facebook, but maybe we'll design some of a similar application, and see how big a problem can get just starting from a small acorn of an idea.

The huge question
A while ago I saw a question on Stack Overflow about building a "basic social application."  I'm summarizing a lot, but essentially the original poster had spent some months (or more) learning programming and has an idea for a mobile+social application and wants to know "what next?"   Original question here if you want to read it: How to Create a Basic Android Social Service.  The question got closed, probably for a variety of reasons, but mainly because it's a huge spectrum of a topic and the answer would not be well-suited for the Stack Overflow format.  I don't know the original poster, but I'm going to assume based on some of his or her statements that the poster is somewhere in the novice category. So a bunch of this article is going to remain high level.

So let's start
What is the application going to do?  (I'll reread the question again...)

• user signup and login (personalization!)
• user profile (so we can get to know them just like FB)
• make posts (what did you have for breakfast?)
• view posts (everything is public for now)
• special sauce (we'll come up with this later)
• use "standard practices"

This list by itself is the basics. This list is also potentially huge. Where do we even begin?  Just based on this list we know a few things.  It's going to be some kind of client-server application.  Although it might be fun to come up with a peer-to-peer social-network application... that might get kind of weird and non-traditional.  So before we get too far, we need to pick a client and a server.  I know you might not care about that and just want to start building the application, but we're building this mythical company from the ground up and somebody needs to decide.

Client platform
Android. That's what the question was tagged as so that's what we'll use.  We could just make this a web application that renders superbly on mobile, but true client applications still (as of this writing) seem to have a quicker, more natural response.

Server platform
This could be anything really but we have to narrow it down somehow.  Since I'm most familiar with Java, we'll stick with that (yes, .Net is a great platform, but we have to narrow the field somehow).  Even after we've selected Java there are still several choice we have to make as we'll soon see.  First, are we going the "traditional" Java EE route using the Java language or when I said "Java" above did you read that as "anything-that-runs-on-the-Java-JVM"?  If the latter, then there are even more choices of language: Groovy/Grails, JRuby, Scala, etc.  For now let's stick with standard JavaEE.

Hosting
Are we going to host this on our own servers (either in-house or remote)?  If not, we can host this in the cloud on  Google App Engine, Amazon EC2, VMware vCloud, Rackspace Cloud, Red Hat Cloud, or probably a dozen others. Most of the choices here will be compatible enough with how we're going to build this application that it almost doesn't matter. Note: Google App Engine is going to be the most different from the rest listed above, but for argument's sake we'll design our application such that it will work almost the same there.  There will no doubt be some economic impact, but we have a lot of mythical dollars so we'll ignore cost.  For now let's just say we'll use some generic cloud provider simply because I don't want to buy the hardware myself.

Application Server
What Java EE application server are we going to use?  I know you probably have your favorite, and for very good reasons. I have a favorite also. Just pick one out of the following list (they're listed in alphabetical order to avoid personal bias): Geronimo, GlassFish, JBoss, Jetty, Tomcat, WebSphere, probably a bunch of others, too.  There are a lot of choices here but just like the hosting, it almost doesn't matter.  Sure, one of them is going to perform better than another one, but if we write our application correctly we won't be tied to any particular app server (maybe just one or two config files different.)  Another note: if you picked Google App Engine from the previous paragraph, you don't need to select which Java EE app server.

Database
What database are we going to use?  The original poster tagged the question with "mysqlite" but may have conflated SQLite (for Android) and mysql into one.  For on-device storage for the Android client portion of our app, we'll of course use SQLite, but on the server we could use anything.  MySQL, Oracle, SQLServer, Postgress, DB2, HSQLDB, CouchDB, MongoDB, Amazon SimpleDB, etc.  Seriously the list is too huge and there are a lot I'm leaving out unintentionally, except for Sybase which I'm leaving out intentionally, but don't ask me why, it's personal.  A big choice here is whether to use a "traditional" relational database, or a non-relational one.

What's next?

We've barely scratched the surface and haven't actually designed the application yet!  Where do you want to go in the next installment?  Probably the server side.  Stay tuned.  Also, please leave comments if you have suggestions on things I left out or other topics I should cover.

See Also
For additional reading: Teach Yourself Programming in Ten Years

Immortalized in the Commit Logs

My name is now logged forever in the commit comments of a pretty Awesome Project!

Have a look at the commit log for yourself (look for my name next to "NPE").

Here's how it happened: One quiet evening in June I was looking at some code diffs of that Awesome Project (don't ask why) and saw something that didn't look quite right. Keep in mind all I was viewing was the diff so I didn't see the full context of the code. Here's the snippet I noticed:

if (varTable == null) {
    if (DEBUG) System.out.println("resizing from " + varTable.length + " to " +  getMetaClass(). getRealClass(). getVariableTableSizeWithObjectId());
    // lots more code...
}

Do you see the NullPointerException? The code just checked if varTable was null, then proceeds to dereference it. That just can't be right. I double-checked it and then finally looked at the whole file. Then I saw it, the DEBUG flag is set to false way up at the top of the file. That's why the lurking NPE never causes a problem. So what's your opinion? Is it a real bug? The code will never get executed so it's not really a problem. Is a NullPointerException still a bug if it's located in a block of unreachable code?

In any case, I thought I should at least alert @Headius, the guy who wrote it. The fix went in the next commit and he put my name in the comments! If I ever see @Headius speak at a conference I'll see if he'll autograph the commit log.

a reasonable implementation of prime number checking

Once again prime numbers on my mind, but this time I won't write the program in BASIC like I did 20+ years ago...

So here we go, a quick implementation in Java, and yes, I'm assuming non-negative input:

public static boolean isPrime(int n) {
    if (n==0 || n==1) return false;  // careful!
    for (int i=2; i<(n/2)+1; i++) {
        if (n % i == 0)
            return false;
    }
    return true;
}

Originally I ran the for loop all the way up to n, but half-n will suffice. There's another efficiency improvement that can be made to the algorithm, I didn't have it in my head until reading it on StackOverflow. Also there's some interesting reading material on Wolfram MathWorld.