Brad's Brain Matters

This Old Code Still Compiles

noreply@blogger.com (Brad Parks) — Thu, 12 Apr 2012 17:01:00 +0000

Message to my younger self, in case you time travel and find this blog: You won't realize it while you're working on that project, but you're learning life lessons about software development that are going to apply to dozens of future projects, so pay attention.

Recently I found my almost-two-decade-old C code from the first "big" software project that I created (about 2000 lines of code in 20 files). It was the first "real" product I made that actually got used. Last night, after I installed the termcap and ncurses libraries on my Linux box, the code compiled with no source or Makefile changes (although there were a lot more compiler warnings than I remember!).

Looking back there are at least two things I learned on that project back then that are timeless. Of course I didn't know it then, but it was early training for real-world projects.

You'll often need to learn something totally new just to get the job done.
Users really like output. And they like it a certain way.

Background: the Timeclock Project

The project was a terminal-based timeclock program for the students who worked in the college computer labs. The 25 student T.A.'s who worked the lab would work varying length shifts and this program handled clocking in/out, saving the data, and it generated timesheets for the computer lab manager. It didn't have a ton of features, but I was pretty proud of it.

Frequent Compilation

I used a basic terminal editor (vi) to create it; thus there was no syntax highlighting and no red squiggly lines to point out syntax errors. My approach was to execute a 'make' from within vi after writing only a few lines of code. Next time your fancy IDE lets you know about syntax or API mistakes, stop for a second and try to remember what it was like before these features.

A Library For the UI

Does anyone here remember curses? That C library made it possible to create a terminal program that performed basic text UI functions. Until this project I had only done basic print-to-screen and read interaction. So just to make a decent interface I had to learn this library. Have I used this library since then? Not until tonight when I installed it on my Linux box to make this compile.

Specific Output

Next, the college's business office had a specific process for handling student timesheets. Paper timesheets of course. The timesheet form they used had been copied and re-copied until it was a blurry but still mostly readable form. How was I going to make output look like that? Well, you may or may not believe this, but sometimes people produced output before PDF existed. We had laser printers - this wasn't the dark ages after all. So I went to Barnes & Noble and bought a book on how to create a PostScript file and over the next two days I learned PostScript and make a form that looked just like the college business office's timesheet form. It was such a good-looking form that the business office started using that as the master copy. A couple years later when I was back on campus I noticed they were still using it, except by then it was a photocopy of a copy of a copy...

A Couple Short Semesters of Use

The timeclock software didn't last longer than a few short semesters and it never expanded to other departments beyond the computer lab TA's. There were a few requested changes to it along the way (user management, supervisor editing for tardy employees, etc). I had high hopes for its continued use, but alas... I had graduated and they eventually stopped using it.

Those Lessons I Mentioned Earlier

1. Learn Something Totally New to Get the Job Done
Every new project could necessitate a new technology or library that you've never tried before. (For example, I've been using Java for a long time and I've never used it to communicate to hardware through a USB port.) I'm not saying you should go out of your way to try to use something new on every project; it will just happen more often than you might expect. Approach this carefully, analyze your options, and get some recommendations from others if it's a totally new technology for you.

2. Users Like Output
Like it or not your users often probably don't want to use your program. Your users want their output and your program is (hopefully) the most effective means for them to accomplish that goal. Depending on the software and the user-base, (specifically I'm talking about PDF or report-generating software), there are going to be times when the output looks perfect to you. Then a user will email you and want to know why a word is in the wrong spot or something doesn't line up just right. Listen to your users (and especially listen to your testing group). Take pride in making your output perfect. If you're lucky it will be

Rooting an old Samsung Intercept

noreply@blogger.com (Brad Parks) — Sun, 25 Mar 2012 18:51:00 +0000

A friend gave me her old Samsung Intercept though she didn't know why I'd want her "crappy old phone." And I must say, with all the bloatware that Sprint has loaded up on this seriously underpowered phone, it was indeed a pretty crappy experience.

From what I've read online, there were a lot of complaints about the Intercept, but I'd place most of the blame for its poor performance squarely on Sprint. Before I started messing with it I should have taken an inventory of all the services that Sprint had running in the background. There were a lot.

Anyway, this was my first attempt rooting a phone and putting on a custom ROM. I'm still new at this and I figured I should take a few notes before I forget everything I've done to it.

Disclaimer: Blah, blah, blah, you might destroy your phone; don't blame me if you do.

Overview:

Install custom "Recovery" tool (CM01 Custom Recovery)
Install custom ROM (essentially the OS the phone runs)

I think of the "Recovery" mode on a phone as akin to the BIOS on a PC. Depending on the phone, you hold down a few different buttons and press the power button to enter Recovery mode. For the Intercept, hold the Volume-Down and Talk buttons when you power on (this usually takes me a few tries to get the timing right. If the progress bar goes past the Sprint logo at the bottom it didn't work). The reason we need to replace the Recovery mode is that the one preinstalled on the phone does some kind of check before installing a ROM update to make sure it's "approved", i.e. that it comes from Sprint. So we replace it with a better one, in this case CM01 Custom Recovery, which originated with the CyanogenMod folks.

There is a wealth of information on the SDX-Developers forums. The guide I used was one titled "[GUIDE] - Install CM01 Custom Recovery using SWUpgrade Tool." The steps involved installing a program "SWUpgrade" onto a Windows PC and downloading a few other things. I'm not going to repeat it all here but be careful that you don't try to install the Samsung Moment's software on top of your Samsung Intercept. That won't be good for anyone. There are other methods of getting root and then installing the CM01 Recovery, but that's what I used. (Another thread says you can just download an apk from somewhere and gain root that way. I didn't try it.)

So anyway, that gets the CM01 Custom Recovery installed, and from there I was able to install and try a few different ROM's, with varying degrees of success. (Note: if you see references to DL05 or EC07, that's apparently two different kinds of Intercept: DL05=Sprint, while EC07=VirginMobile). I just copied the appropriate zip file to the SD Card in the phone, selected it from the Recovery menu, and installed. (Sometimes doing a wipe beforehand to make sure the phone is fresh).

ROMs I've tried so far
(mostly found from another SDK Developers thread on ROMs for the Intercept):

inxane0381intercept - this didn't work for me at all; the screen was dark and the phone just sat there and buzzed/vibrated in a dot, dit-dit-dit pattern. Kinda scary and thought I'd bricked the phone, but fortunately got back to the beginning.
Stock ROM on Crack for DL05 - works great. phone is way more responsive now
Ice Cream Froyo - looks like ICS but still Froyo, partially didn't work for me.
Back to the original FB01 Stock ROM.
Ubuntdroid 6.0 DL05 - which worked okay and had neat graphics but didn't seem particularly fast and apparently has nothing to do with Ubuntu, they just like the name and logo.
Another thread says there's a working ROM of CM6 for Intercept available on 4shared - I'm trying this one now, my fallback will be to use the Stock ROM on Crack since that seems to be the best thus far.

Summary: While this was fun and all, I really don't feel like I did anything special or creative -- I just followed some existing guides, following work done by those who came before me. I think a better challenge would be to build Android from source and see how that goes.

You're Not the First to Have This Problem

noreply@blogger.com (Brad Parks) — Mon, 06 Feb 2012 03:08:00 +0000

This is going to sound like the "compiler bug" problem, but this time it was something very similar. Yes, there are probably some bugs in the JDK. The likelihood of there being a bug in any software increases along with the size of the codebase. (Unsubstantiated claim, but I provide no footnotes with this blog.)

Anyway, when you hit a problem keep in mind that you're never the only person to have encountered this problem. Somebody else has seen it and probably been in a more desperate situation to need a solution.

Every once in a while I'd see some odd stuff going on where a bunch of threads seemed to get stuck. Those threads interacted via a CORBA interface, so I just chalked it up to some CORBA-wackiness and thought I'd try to code a way to kill off those threads when they got stuck waiting for CORBA to finish.

Sidenote: CORBA came from the OMG, which has the most ironically appropriate acronym ever invented, because "it's like, OMG, could there be anything better than a consortium named OMG!?" By the way, do not click over to the talk page on Wikipedia entry for OMG if you're in a quiet professional environment -- I started laughing so much my wife thought there was something wrong with me.

So anyway, I got a recent thread dump (not a stacktrace, since those threads didn't throw any exceptions, they were just waiting) and decided to look around a bit for the answer. Here's a sample of one of the stuck CORBA threads:

"pool-538-thread-1" - Thread t@67441

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <5dfd006a> (a java.lang.Object)

at java.lang.Object.wait(Object.java:485)

at com.sun.corba.se.impl.transport.CorbaResponseWaitingRoomImpl.waitForResponse(CorbaResponseWaitingRoomImpl.java:140)

at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.waitForResponse(SocketOrChannelConnectionImpl.java:1084)

at com.sun.corba.se.impl.protocol.CorbaMessageMediatorImpl.waitForResponse(CorbaMessageMediatorImpl.java:253)

at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.marshalingComplete1(CorbaClientRequestDispatcherImpl.java:362)

at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.marshalingComplete(CorbaClientRequestDispatcherImpl.java:336)

at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.invoke(CorbaClientDelegateImpl.java:129)

at com.sun.corba.se.impl.corba.RequestImpl.doInvocation(RequestImpl.java:309)

at com.sun.corba.se.impl.corba.RequestImpl.invoke(RequestImpl.java:230)

- locked <6dae8354> (a com.sun.corba.se.impl.corba.RequestImpl)

[etc., etc...]

I stumbled around the Interwebs for a bit and happened upon the inevitable similar stack traces and mentions in a forum or two or three, but none of them had the answer. Then I expanded my search a bit and found an existing item (7016182) in the Java bug database... submitted about a year ago and a status of "Cause Known" but now what? So I searched on the related bug ID numbers and found yet more info in number 7046238. Yay! Finally an answer! It says it's been fixed in JDK release 6u27 which I'm pretty sure is 1 update newer than what we were running.

That's good news, but it still needs to get tested. In the meantime, I compared the source of the affected JDK class CorbaClientRequestDispatcherImpl with the the fixed version. Yep, just as the bugfix note said, the fix is a change to how the synchronization is handled and it makes me glad I didn't have to try to figure out how to fix it. In a way it's comforting to know that even the experts sometimes make mistakes.

Anyway, that was my adventure this weekend. What did you do?

Modeling Intangible Many-to-Many Relationships

noreply@blogger.com (Brad Parks) — Thu, 02 Feb 2012 09:17:00 +0000

Modeling intangible things in a data model takes practice. Here's an example I've used and choices made along the way.

Visible and Real

Set aside the database "join table" for a moment and consider how you represent your primary entities in code. Sometimes it's easy to see how the concepts should map to database tables and Java classes -- especially when the concepts are relatively visible in the real world.

Let's take, for example, my favorite model: Creature - Skill - Achievement.

Creature is obviously a visible "thing" that exists in the real world. Skill and Achievement aren't quite as visible except in a temporal sense, but at least while you're watching a Creature swim (or whatever other activity), you can see that it's happening with some level of skill. An achievement like "first place" or "swam 10 laps" may be represented by an award of some kind but there the visible thing is just a reminder of an achievement, it's not actually the achievement itself, but somebody saw it happen so at least for a moment it was real. So really this one isn't so hard to wrap our minds around. We can have Java classes with a 1-to-1 correlation with the database tables and it all pretty much makes sense.

Invisible Things

But when entities represent something intangible the mapping isn't as clear. A few years back this situation cropped up in a development effort where I was a (pseudo) dev lead. Our application was going to deal with People and Decisions. In this instance a "Decision" could be something like a "Decision to Exercise Regularly." Each Person could have many Decisions. Each Decision could be made by several People. Thus, a many-to-many relationship. So how do we model this?

or translated into something nearly equivalent...

The database folks already had the tables set up as Person, Decision, and PersonDecision which followed their standard naming conventions. And, at least with a traditional database, this is how a many-to-many relationship is created.

Option 1.
Should we stick with a 1-to-1 mapping from database table to Java class? In that case a Person object would have a collection of PersonDecision objects and some field there in the PersonDecision class would show what kind of Decision was made by the Person. This option has the benefits of being a simpler implementation for mapping.

Option 2.
However, it could be argued that each Person object should directly have a collection of Decision objects, with any necessary metadata from the database's PersonDecision table as fields in the Decision class. This option has more object-oriented beauty about it and avoids the awkward PersonDecision name.

Consider the Usages (and the kittens)

We could have talked about it for days but there was a lot of work ahead of us so I went with my gut and chose for us to go with the first option (though there was some further discussion expressing respectful disagreement). The major factor for me is that there was additional data attached to both Decision and PersonDecision (such as EventDate). Also, there were some usages within the application that involved only Decision and not Person or PersonDecision. So even if we had merged the data from the two tables into one class, it would still have required a Template Decision class representing an archetype of a Decision separate from a Person. Thus we would have still needed an additional class, just perhaps with a different name, putting us back into situation we started with plus some additional complexity.

Who really knows whether or not my choice was the correct one. It's the one we went with on that project and it didn't feel like we had to jump through a lot of hoops to make things work. Revisiting it, maybe it would be nice to implement it both ways to compare the merits of each option more closely. But that's a topic for a different post.

Build Your Own Facebook - Part 1

noreply@blogger.com (Brad Parks) — Sat, 24 Sep 2011 02:05:00 +0000

Here's a billion mythical dollars - let's design the next popular mobile/social application. I don't know how long this series (!) of posts will last but I'll at least start with this one. We're not actually going to build a Facebook, but maybe we'll design some of a similar application, and see how big a problem can get just starting from a small acorn of an idea.

The huge question
A while ago I saw a question on Stack Overflow about building a "basic social application." I'm summarizing a lot, but essentially the original poster had spent some months (or more) learning programming and has an idea for a mobile+social application and wants to know "what next?" Original question here if you want to read it: How to Create a Basic Android Social Service. The question got closed, probably for a variety of reasons, but mainly because it's a huge spectrum of a topic and the answer would not be well-suited for the Stack Overflow format. I don't know the original poster, but I'm going to assume based on some of his or her statements that the poster is somewhere in the novice category. So a bunch of this article is going to remain high level.

So let's start
What is the application going to do? (I'll reread the question again...)

user signup and login (personalization!)
user profile (so we can get to know them just like FB)
make posts (what did you have for breakfast?)
view posts (everything is public for now)
~~special sauce~~ (we'll come up with this later)
use "standard practices"

This list by itself is the basics. This list is also potentially huge. Where do we even begin? Just based on this list we know a few things. It's going to be some kind of client-server application. Although it might be fun to come up with a peer-to-peer social-network application... that might get kind of weird and non-traditional. So before we get too far, we need to pick a client and a server. I know you might not care about that and just want to start building the application, but we're building this mythical company from the ground up and somebody needs to decide.

Client platform
Android. That's what the question was tagged as so that's what we'll use. We could just make this a web application that renders superbly on mobile, but true client applications still (as of this writing) seem to have a quicker, more natural response.

Server platform
This could be anything really but we have to narrow it down somehow. Since I'm most familiar with Java, we'll stick with that (yes, .Net is a great platform, but we have to narrow the field somehow). Even after we've selected Java there are still several choice we have to make as we'll soon see. First, are we going the "traditional" Java EE route using the Java language or when I said "Java" above did you read that as "anything-that-runs-on-the-Java-JVM"? If the latter, then there are even more choices of language: Groovy/Grails, JRuby, Scala, etc. For now let's stick with standard JavaEE.

Hosting
Are we going to host this on our own servers (either in-house or remote)? If not, we can host this in the cloud on Google App Engine, Amazon EC2, VMware vCloud, Rackspace Cloud, Red Hat Cloud, or probably a dozen others. Most of the choices here will be compatible enough with how we're going to build this application that it almost doesn't matter. Note: Google App Engine is going to be the most different from the rest listed above, but for argument's sake we'll design our application such that it will work almost the same there. There will no doubt be some economic impact, but we have a lot of mythical dollars so we'll ignore cost. For now let's just say we'll use some generic cloud provider simply because I don't want to buy the hardware myself.

Application Server
What Java EE application server are we going to use? I know you probably have your favorite, and for very good reasons. I have a favorite also. Just pick one out of the following list (they're listed in alphabetical order to avoid personal bias): Geronimo, GlassFish, JBoss, Jetty, Tomcat, WebSphere, probably a bunch of others, too. There are a lot of choices here but just like the hosting, it almost doesn't matter. Sure, one of them is going to perform better than another one, but if we write our application correctly we won't be tied to any particular app server (maybe just one or two config files different.) Another note: if you picked Google App Engine from the previous paragraph, you don't need to select which Java EE app server.

Database
What database are we going to use? The original poster tagged the question with "mysqlite" but may have conflated SQLite (for Android) and mysql into one. For on-device storage for the Android client portion of our app, we'll of course use SQLite, but on the server we could use anything. MySQL, Oracle, SQLServer, Postgress, DB2, HSQLDB, CouchDB, MongoDB, Amazon SimpleDB, etc. Seriously the list is too huge and there are a lot I'm leaving out unintentionally, except for Sybase which I'm leaving out intentionally, but don't ask me why, it's personal. A big choice here is whether to use a "traditional" relational database, or a non-relational one.

What's next?

We've barely scratched the surface and haven't actually designed the application yet! Where do you want to go in the next installment? Probably the server side. Stay tuned. Also, please leave comments if you have suggestions on things I left out or other topics I should cover.

See Also
For additional reading: Teach Yourself Programming in Ten Years

Immortalized in the Commit Logs

noreply@blogger.com (Brad Parks) — Thu, 04 Aug 2011 03:39:00 +0000

My name is now logged forever in the commit comments of a pretty Awesome Project!

Have a look at the commit log for yourself (look for my name next to "NPE").

Here's how it happened: One quiet evening in June I was looking at some code diffs of that Awesome Project (don't ask why) and saw something that didn't look quite right. Keep in mind all I was viewing was the diff so I didn't see the full context of the code. Here's the snippet I noticed:

if (varTable == null) {
    if (DEBUG) System.out.println("resizing from " + varTable.length + " to " +  getMetaClass(). getRealClass(). getVariableTableSizeWithObjectId());
    // lots more code...
}

Do you see the NullPointerException? The code just checked if varTable was null, then proceeds to dereference it. That just can't be right. I double-checked it and then finally looked at the whole file. Then I saw it, the DEBUG flag is set to false way up at the top of the file. That's why the lurking NPE never causes a problem. So what's your opinion? Is it a real bug? The code will never get executed so it's not really a problem. Is a NullPointerException still a bug if it's located in a block of unreachable code?

In any case, I thought I should at least alert @Headius, the guy who wrote it. The fix went in the next commit and he put my name in the comments! If I ever see @Headius speak at a conference I'll see if he'll autograph the commit log.

a reasonable implementation of prime number checking

noreply@blogger.com (Brad Parks) — Tue, 29 Mar 2011 02:06:00 +0000

Once again prime numbers on my mind, but this time I won't write the program in BASIC like I did 20+ years ago...

So here we go, a quick implementation in Java, and yes, I'm assuming non-negative input:

public static boolean isPrime(int n) {
    if (n==0 || n==1) return false;  // careful!
    for (int i=2; i<(n/2)+1; i++) {
        if (n % i == 0)
            return false;
    }
    return true;
}

Originally I ran the for loop all the way up to n, but half-n will suffice. There's another efficiency improvement that can be made to the algorithm, I didn't have it in my head until reading it on StackOverflow. Also there's some interesting reading material on Wolfram MathWorld.

Reading for Inspiration and Motivation

noreply@blogger.com (Brad Parks) — Sat, 12 Mar 2011 22:01:00 +0000

I just finished the book Rework by Jason Fried and David Heinemeier Hansson (see also 37signals). It's really inspirational. One of my favorite parts of the book is from page 38.

"What you do is what matters, not what you think or say or plan."

This is the year to release whatever you're working on in your spare time. Even if it's small. Even if you're not sure anyone will use it. Release it. Get it out there now.

Also on the same page

"The original pitch idea is such a small part of a business that it's almost negligible."

That sentence seems even more relevant if you've seen The Social Network.

The book isn't very long. I'm sure there will be some parts of Rework that will hit home for you. When I read this book again next year there will probably be some different parts that speak to me.

What book(s) have you read recently that inspired or motivated you?

Using Javadoc to stay healthy

noreply@blogger.com (Brad Parks) — Thu, 24 Feb 2011 02:00:00 +0000

Do you Javadoc ™ ? IDEs make it easy enough to create skeletal Javadoc comments in your code, so you probably (hopefully) have at least some Javadoc documentation. But when was the last time you actually ran the javadoc tool on your code to look at what it creates? It's been a while since I've run it on the codebase at work so I thought I'd give it a try. In doing so I found a couple nice gems.

Javadoc Gem #1 - Finding Stale Code
I found some stale code thanks to the {@inheritDoc} tag. The output from running javadoc alerted me to the fact that a few implementation classes had the {@inheritDoc} tag but didn't have a corresponding method in their parent (either interface or abstract class.) Since we (almost) never go straight to the implementation class, a quick search through the code found that those methods weren't being used anywhere except in a JUnit test. (So even though we aren't actually using the code at least it's well tested!)

BlahBlahBlahImpl.java:41: warning - @inheritDoc used but 
staleMethod() does not override or implement any method.

Javadoc Gem #2 - Nonstandard Copyright
I learned that the @copyright really isn't a valid javadoc tag. Yes, I actually looked it up in the javadoc reference and it's just not there. I don't remember who the first person on the team was that started the trend of using that, but apparently it's not right. Another quick search of the code showed that we used it 343 times. Hmm, should I just do a global search and replace? Better yet, maybe we can just redefine reality. There's a way to declare a custom javadoc tag and was easy enough to declare the custom javadoc tag in the maven config too.

BlahBlahBlah.java:11: warning - @copyright is an unknown tag.

Lessons Learned
I'm not one to shy away from stating the obvious, so I'll list out some lessons learned from this experience:

Tools are more useful when they're used more than once every 3 years. Yeah, it's probably been that long since I generated and looked at javadoc.
Comment the code as I go. It would be kinda painful to go back now and figure out what all these uncommented methods are doing. Sure, I can probably guess what a method does from the name, but it would be nicer to have some context to go along with it. Apparently at one point we found this to be useful: Set<Long> toLongSet(Collection<? extends Number> numbers)
Find a better way to detect unused code. I'm sure there's some tool out there for this. Got any ideas?

Java 7 Features, the Future can't come soon enough.

noreply@blogger.com (Brad Parks) — Wed, 26 Jan 2011 04:25:00 +0000

If you're like me, perhaps you wish the future was here already. A flying car would be nice and all, but I'll settle for the next version of Java.

For anyone who hasn't seen it yet, here's the planned list of Java 7 features.

The official JSR #336 for Java 7 is worth a read, though it's easy to get distracted by the Fork/Join Framework and veer off to read more about that. And since we're on the topic, if you've never heard about Parallel Arrays, you should learn something about that. (Okay, now I'm just talking to myself).

So even though the future isn't here yet, we can make it arrive sooner by downloading a current binary snapshot release of JDK 7 to kick the tires and try it out.

Let's get started.

Time to Reminisce - Computing Prime Numbers

noreply@blogger.com (Brad Parks) — Mon, 17 Jan 2011 20:28:00 +0000

Set the WABAC Machine for the late 1980s...

What's the first long running program you remember writing? One summer at the high school computing lab I saw the room full of PCs were sitting idle and thought it would be fun to write a program that would run for a long time.

Two math problems came to mind: 1. find prime numbers less than a bazillion, 2. compute pi to a bazillion digits. I chose the first one mainly because it would be relatively easy to verify. After all, if I found the 904th digit of pi was a 2, how was I going to verify it?

The only language I had available to me at the time was BASIC. Don't judge - that's what the school had and that's all I knew at the time. I didn't have much of an algorithm for finding primes, but I had a trusty For Loop and a lot of time. I don't think I have the code any longer, kinda wish I still did so I could try it now. My method was pretty simple: Increment a counter to the next number, start a loop up to 1/2 of that number and see if it divided evenly (quick, tell me what the big O notation of that would be). I'm sure there are better ways than just a loop inside a loop but this was a long time ago... You kids these days with your Google could probably find me a better algorithm.

I knew this program was going to run for days and could be subject to interruption (such as being unplugged by a janitor) but wanted to keep a good record of my results. My immediate choices were to periodically save to a file or send a line to the printer. Saving to the 5.25" floppy would have been fine I suppose, but I chose to use the printer because the progress would be more visible and permanent. After finding 10 prime numbers, my program would send one line to the printer listing them.

It ran for days (maybe a couple weeks) before it finally was interrupted. Every so often the printer sprang to life and added one line of prime numbers. Obviously it was quicker with the smaller numbers because it had smaller numbers. Eventually my program filled about 6 or 7 pages. I don't really remember how far it got, but I do remember being rather proud of the result. I taped the output up on the wall in my bedroom - I should call my mom and see if it's still there.

On the next episode of "Time to Reminisce" maybe I'll talk about the first program I wrote for somebody else.

Why is my PreparedStatement query slow? (can I blame Sybase?)

noreply@blogger.com (Brad Parks) — Mon, 10 Jan 2011 05:20:00 +0000

This week I was asked to take a look at a batch application project that was having problems. The project uses Spring Batch to run a sequence of SQL statements and a variety of other things. At one step a pretty simple SQL query would just sit there, no response coming back, until something would just time out and we'd get this exception:

java.sql.SQLException: JZ006: Caught IOException: java.net.SocketTimeoutException: Read timed out

A socket timeout could be caused by at least four different things, so let's narrow it down. One of the first questions I always have with database related problems like this: "Is there anything wrong with the query?"

The query wasn't very complicated, it was a simple, one-table query and no joins. The log showed this as the query right at the SocketTimeoutException exception.

SELECT
        STUFF_ID, 
        STUFF_DESC, 
        MORE_STUFF, 
        ? AS BATCHID
    FROM
        MY_STUFF_TABLE
    WHERE
        STUFF_UPDATE_DT > ?

Notice the question marks are bind parameters for a prepared statement that gets filled in at runtime.

After trying this out in my favorite database tool, I saw that the query was fast, even with 20 million rows. So no problem there.

I double-checked the driver version and that all seemed fine. (There was a database upgrade recently from Sybase 12.5 to Sybase 15). Checking the driver version is what we like to call "low-hanging fruit" - easy to check and easy to fix - so maybe should have checked that first.

When there are a lot of different components involved I like to reduce the problem down to its most basic elements. So I got back to basics and whipped up a tiny amount of Java using old-school JDBC. (Remember how we used to connect() and prepareStatement() a decade ago?) So repeating that exercise with the appropriate SQL and making sure to use a prepared statement, the end result was a slow query! Zoinks! I replaced the prepared statement with the full query (i.e., no bind parameters) and found it fast again, just like when running it with DB Visualizer.

Okay, now we know it's got nothing to do with the framework.

From past experience I know that sometimes there can be screwy side-effects to optimizations that a database server performs with bind params. I'm not a Sybase expert so I can't tell you what's going on internally, but often a database driver has some way of flipping a toggle to alter the optimizing behavior. So I went off to check some documentation for jConnect.

The two most interesting settings were LITERAL_PARAMS and DYNAMIC_PREPARE since they were somewhat related to substituting values in a prepared statement, etc. A quick check of the documentation said we could set this in the connect string like so:

jdbc:sybase:Tds:127.0.0.1:9999/exampledb?LITERAL_PARAMS=true

I dropped that into my simple sample code and it was fast like it was supposed to be. Yay! Problem solved, let's throw this fix into the larger application and see if we get the same result. Sure enough, things worked better than they did before. We'll celebrate for 15 minutes and move on to the next problem.

Jetty and Maven error: Cannot find setter nor field

noreply@blogger.com (Brad Parks) — Fri, 10 Dec 2010 01:59:00 +0000

Just this week I was trying to use jetty-maven-plugin to run my web application and getting the following error:

[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to configure plugin parameters for: org.mortbay.jetty:jetty-maven-plugin:7.0.0.pre5
Cause: Cannot find setter nor field in org.mortbay.jetty.plugin.Jetty6PluginWebAppContext for 'jettyEnvXml'

The command line I was using was just

mvn jetty:run

And I was following the <webAppConfig> documentation for configuring the Jetty external XML file (<jettyEnvXml>) in my pom.xml

Turns out that the version I'm using calls that setting <jettyEnvXmlFile>. So try that and let me know how that works for you.

Document Your Data, it's the Most Reusable Component

noreply@blogger.com (Brad Parks) — Wed, 08 Dec 2010 01:16:00 +0000

"Metadata is the key to reusability of data and analysis." - DW 2.0 book, page 110.

Recently I've been reading DW 2.0: The Architecture for the Next Generation of Data Warehousing (Inmon, Strauss, Neushloss) and while reading chapter 4 about Metadata it got me thinking -- "Where do we maintain the metadata about our data model?" (possible metadata could include: Definition, Usage Locations, Owner, Synonyms, etc.) Often, the most documentation I see is stored as comments in the code.

Overheard: "I have a CONTACT table in my database, but I don't remember what it's for."

As an application developer I see things through code-colored glasses. Don't get me wrong, I love data, but I love the application first and foremost. The functionality, the usability, the algorithms, the behavior, the bells, the whistles, etc. Thus, my approach when looking at the entire application, I see the code first. For example, if I have an application that deals with People I probably have an entity in the domain model named "Person" or "Contact" that's usually stored in a database, often in a table by the same or similar name. It's only natural for me to include Javadoc comments in the class that contains a richer description of what we mean when we talk about that concept. (See also "ubiquitous language" as mentioned in Domain-Driven Design) So is that the best place for the definition?

Reflection Point: does the data exist to support the application? or does the application exist to support the data? Is your organization application-centric or data-centric?

If your company is application-centric and has only a single application, placing the metadata/documentation of the full meaning of an entity inside the code is probably fine. But more than once I've seen an entire application rewritten. Sometimes it happens a component at a time, but still not a line remains of the original code. So if the only metadata information for an entity is in the code, that information that will need to get carried along with each potential rewrite. This then isn't always the best place for the data definitions.

Also, think carefully about your application and data - Do you really have only one application using that data? Often there are several applications using or reporting on the data. (Perhaps your company really is data-centric and you're looking at it only from the perspective of the code.) Wouldn't it be nice if all the applications shared the same definitions? Oracle, MySQL, and probably other databases provide for the ability to attach comments to tables (or even at the column level). Modeling tools like ERwin also work well for storing table definitions (including maintaining Logical as well as Physical representations of the model). Depending on how much detail you need there are other tools as well which can be used for documenting (note to self: check out CASEwise and Rochade).

So how well does this level of documentation fit with Agile methodology? I'd suggest adding it to your team's definition of "done." A new feature isn't done until and unless the corresponding data model's definition has been updated.

Open Letter to Radio Personalities, Audio Ad Writers, and Voice Talent

noreply@blogger.com (Brad Parks) — Thu, 14 Oct 2010 03:02:00 +0000

Dear Radio Personality, Ad Writer, and/or Voice Talent,

When reading or writing a radio advertisement for a company with a web address keep in mind the following rule:

There is no such thing as a backslash in a website address.

It is always a slash. ALWAYS. I'm serious. No exceptions. It's always a slash. That's the symbol down there on the keyboard with the question mark. Seriously. Look it up if you don't believe me. It's called a slash. Example: "double-you double-you double-you dot wkrp dot com slash news"

Let me go over this one more time:

This is a slash /
This is a backslash \
The slash / is used in web addresses
The backslash \ does not work on the Internet
The slash / makes you sound smart
The backslash \ makes you sound not-so-smart

Honest. I wouldn't make this up. Go ask Tim Berners-Lee and see what he calls it. I don't care what your boss or station manager or scriptwriter says - it's a slash.

Don't make me call in on the request line and complain. Again.

Sincerely,

-Brad Parks

Remote Desktop: Increase Maximum Number Connections, but That's Not What I Really Needed

noreply@blogger.com (Brad Parks) — Tue, 12 Oct 2010 00:53:00 +0000

"The terminal server has exceeded the maximum number of allowed connections." -Remote Desktop, Windows Server

That's the error message I got today trying to use Remote Desktop to get into one of our servers. All I wanted was to deploy some changes to our DEV server. How can there be too many connections? We're a small team and there are only 3 of us that regularly log into the DEV box. The other 2 guys are +6 timezones away from me and have surely left the office by now. So that means they must have left themselves logged in. I really didn't want to wait till the next day for the other guys to get into the office.

So how can I increase the maximum number of allowed connections? That's what I thought I wanted. (I don't want to spoil the ending of this post but I actually solved it a different way). A quick search of the Interwebs informed me that two connections is the maximum unless extra $tuff is installed on the server.

Then I happened upon this question on ServerFault.com about somebody else who wanted to "Increase number of allowed remote desktop connections" so at least I'm not alone in wanting this. One of the answers to that question is to use the qwinsta command to Query for remote sessions and then rwinsta to Reset a remote session.

Here is my example usage of each of these commands on server DEVSRV99:

Query:

C:\> qwinsta /server:DEVSRV99.bradsbrain.com
 SESSIONNAME       USERNAME                 ID  STATE   TYPE        DEVICE
 console                                     0  Conn    wdcon
 rdp-tcp                                 65536  Listen  rdpwd
                   larry                     2  Disc    rdpwd
 rdp-tcp#72        curly                     3  Active  rdpwd

Reset connection ID 3:

C:\> rwinsta 3 /server:DEVSRV99.bradsbrain.com

After performing the reset (rwinsta) command the number of allowed connections was decreased (as expected). I was then able to connect via Remote Desktop.

So, what can you do if you see this "Access Denied" message when you try the qwinsta or rwinsta command?

Error opening Terminal server DEVSRV99.bradsbrain.com
Error [5]:Access is denied.

The reason for this error is because the user you're trying to run the command as hasn't authenticated to the machine we want to query/reset. And these two little programs don't bother to prompt for the credentials. You can quickly authenticate to the necessary server or domain by putting the server name into Windows Explorer and attempt to browse files on that machine - (\\DEVSRV99.bradsbrain.com\C$) - that should present you with the appropriate username/password dialog.

Visual Studio wdproj project type not supported

noreply@blogger.com (Brad Parks) — Sat, 04 Sep 2010 04:42:00 +0000

The problem: Today's error message is brought to you by a fresh install of Visual Studio 2008. I installed it and tried opening and building an existing project.

'C:\work\MyLovelyProject.wdproj' cannot be opened because its project type (.wdproj) is not supported by this version of the application.

What's wrong: I forgot to install the "Web Deployments Projects" add-in. (similar to Eclipse plugin I guess).

How to fix: Search for "web deployment project plugin visual studio 2008", download, install, and try again.

Referenced file contains errors OR how XSD is killing me

noreply@blogger.com (Brad Parks) — Wed, 11 Aug 2010 00:38:00 +0000

Quick fix for the impatient: you and I have something in the schemaLocation definition in the applicationContext file that doesn't mesh. Check to make sure the versions all match.

Now for the details...

Recently I switched to the SpringSource Tool Suite as my IDE, so far so good. I checked out an old project which I'm sure had no errors before. Now the Problems view tells me:


Referenced file contains errors (jar:file:/C:/springsource/sts-2.3.2.RELEASE/plugins/org.springframework.beans_3.0.1.RELEASE-A.jar!/org/springframework/beans/factory/xml/spring-beans-2.5.xsd).  For more information, right click on the message in the Problems View and select "Show Details..."

This error looked kinda suspicious to me -- it just doesn't make sense that an XSD file from Spring contains errors. I must be reading that wrong. Well, I'll click through to Show Details...

Repeated for the PNG impaired:

The errors below were detected when validating the file spring-beans-2.5.xsd via the file applicationContext.xml. In most cases these errors can be detected by validating spring-beans-2.5.xsd directly. However it is possible that errors will only occur when spring-beans-2.5.xsd is validated in the context of applicationContext.xml.

I think the better clue in the Error Details is the part that says Duplicate attribute uses with the same name and target namespace which to me indicates that my applicationContext.xml is using some attribute incorrectly. So once again we look at the headers:

<beans xmlns="http://www.springframework.org/schema/beans"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:tx="http://www.springframework.org/schema/tx"
        xmlns:jee="http://www.springframework.org/schema/jee "
        xsi:schemaLocation="
            http://www.springframework.org/schema/beans 
            http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
            http://www.springframework.org/schema/tx
            http://www.springframework.org/schema/tx/spring-tx-2.5.xsd
            http://www.springframework.org/schema/jee 
            http://www.springframework.org/schema/jee/spring-jee-2.0.xsd"
 default-lazy-init="true">

Yeah, this file was started a couple years ago. Not sure how we haven't noticed this before (maybe once again we had validation turned off by default), but if you look carefully you see a mix of versions 2.0 and 2.5. Pick one, make them the same, and the Validation error should go away. Why couldn't the error message have said that in the first place?

LifecycleException Failed to invoke lifecycle phase "start" on object: SedaService

noreply@blogger.com (Brad Parks) — Tue, 27 Jul 2010 00:12:00 +0000

Today's stacktrace:

org.mule.api.lifecycle.LifecycleException: Failed to invoke lifecycle phase "start" on object: SedaService{BookService}
 at org.mule.lifecycle.DefaultLifecyclePhase.applyLifecycle(DefaultLifecyclePhase.java:276)
 at org.mule.lifecycle.DefaultLifecyclePhase.applyLifecycle(DefaultLifecyclePhase.java:128)
 at org.mule.lifecycle.GenericLifecycleManager.firePhase(GenericLifecycleManager.java:84)
 at org.mule.DefaultMuleContext.start(DefaultMuleContext.java:179)
 at org.mule.config.builders.MuleXmlBuilderContextListener.initialize(MuleXmlBuilderContextListener.java:83)
 at org.mule.config.builders.MuleXmlBuilderContextListener.contextInitialized(MuleXmlBuilderContextListener.java:58)
  [...]

The solution turns out to be a config problem on my part, just like 90% of 'em. Lately I've been leaning towards contract-first development, so we have a WSDL that I specify in my Mule config:

[...]
  <cxf:inbound-endpoint serviceName="BookService"
      address="http://localhost:44080/services/v1/BookService"
      serviceClass="com.bradsbrain.atlas.generated.BookService"
      wsdlLocation="BookService.wsdl"
      synchronous="true" applyFiltersToProtocol="true" applySecurityToProtocol="true" applyTransformersToProtocol="true" >
      <property key="namespace" value="http://atlas.bradsbrain.com/services" />
  </cxf:inbound-endpoint>
  [...]

And here is the part near the bottom of my WSDL that I had to fix. This part is correct, i.e., after the fix, because I didn't want to include a big section of incorrect XML.

[...]
  <wsdl:service name="BookService">
    <wsdl:port binding="tns:BookServiceBinding" name="BookServicePort">
      <soap:address location="http://localhost:44080/services/v1/BookService"/>
    </wsdl:port>
  </wsdl:service>

The part that I had incorrect is the value of the name= attribute of the wsdl:port tag. I used to have it as name="BookService" and when I switched to CXF 2.1.5 that's when I got the LifecycleException you see above. Changing it to name="BookServicePort" fixed it.

But it worked with the older version of CXF! (I wanna say 2.1.2)

This happened right after we upgraded our version of Mule which as a side effect also shifted us to a newer version of CXF. I upgraded to the Enterprise version of Mule (2.2.4) and a not-even-new version of CXF (2.1.5).

Does anyone want to sift through the CXF version history documents with me and see when, where, and why this changed?

The matching wildcard is strict, and doesn't like you.

noreply@blogger.com (Brad Parks) — Tue, 18 May 2010 15:30:00 +0000

Recently I saw the following error in Eclipse on a recently imported project:

cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'tx:advice'.

Solution? Check namespace and schema location declarations at the top of the file. Here are mine:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:aop="http://www.springframework.org/schema/aop"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:tx="http://www.springframework.org/schema/tx"
     xsi:schemaLocation="http://www.springframework.org/schema/beans
                         http://www.springframework.org/schema/beans/spring-beans.xsd
                         http://www.springframework.org/schema/aop
                         http://www.springframework.org/schema/aop/spring-aop-2.5.xsd
                         http://www.springframework.org/schema/context
                         http://www.springframework.org/schema/context/spring-context-2.5.xsd">

Notice anything missing? It turns out I was missing the schemaLocation for the "tx" namespace (it was listed as a namespace but had no schemaLocation).

        http://www.springframework.org/schema/tx
        http://www.springframework.org/schema/tx/spring-tx-2.5.xsd

Putting that in solved the problem.

Someday I'd like to join onto a new project team where the code compiles cleanly after checkout and is error and warning-free in my IDE. "Just turn off validation for that project" is NOT a valid solution.

Java default values for objects and primitives

noreply@blogger.com (Brad Parks) — Thu, 22 Apr 2010 00:52:00 +0000

or Stuff I Usually Don't Think About #1

This post might be (should be) basic for you Java readers, but it's the kind of thing I rarely think about, with the reason why explained below. So just bear with me a minute or feel free to skip this one, Mr. Know-it-all.

Consider the following Java code and guess what the output will be:

    public void usefulMethod() {
        int total;
        String description;
        System.out.println("The total is: " + total);
        System.out.println("Description length: " + description.length());
    }

According to the Java tutorial "Fields that are declared but not initialized will be set to a reasonable default by the compiler." And that's the way it is for fields, but what about for local variables as in the code above?

Go ahead and paste it into your IDE now and you'll know why I rarely think about it. It's because it won't compile! The IDE points out the error; the programmer adjusts the code and doesn't even think about it (or at least I never do anyway). Of course it's bad practice to not assign a value, and of course we all assign some sensible default without thinking, but that doesn't mean it shouldn't be considered at least once. There, now that you've thought about it you can go back to your Grande Affogato No Whip Mocha Frapuccino and get on with your day. Read just a little further if you want the nitty gritty details of why it's like this.

Well, as always, look to the Java Language Specification (JLS) for the official reason. According to the "Rule of Definite Assignment" in the Java Language Specification, there must be a compile-time error when encountering an uninitialized local variable. So keep this in mind when you're writing your next Java compiler.

Also note, whoever wrote the Java Compiler error message for this must have been from Minnesota... note the overly-polite, non-confrontational phrasing:

variable description might not have been initialized

In Minnesota, that's how we tell somebody they're wrong.

invalid content was found starting with...

noreply@blogger.com (Brad Parks) — Mon, 19 Apr 2010 23:32:00 +0000

Eclipse schema validation fails on a mule config file. It's really kindof annoying to have ugly schema validation errors. Here's what the error looks like in the problem view:

cvc-complex-type.2.4.a: Invalid content was found starting with element 'component'. One of '{"http://www.mulesource.org/schema/mule/core/2.2":abstract-component, "http://www.mulesource.org/schema/mule/core/2.2":outbound, "http://www.mulesource.org/schema/mule/core/2.2":async-reply, "http://www.mulesource.org/schema/mule/core/2.2":abstract-exception-strategy, "http://www.mulesource.org/schema/mule/core/2.2":abstract-service-threading-profile, "http://www.mulesource.org/schema/mule/core/2.2":abstract-queue-profile}' is expected.

After just a bit of searching, I found this helpful and quite detailed information:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=284272

The quick fix is to go into Eclipse Preferences -> XML -> XML Files -> Validation and turn off the checkbox for "Honour all XML schema locations" (see the link above for more details)

With that cleared up I'll hopefully be able to get some work done now.

Convert MySQL table from MyISAM to InnoDB

noreply@blogger.com (Brad Parks) — Sun, 10 Jan 2010 23:25:00 +0000

AS A developer using MySQL I WANT to convert a table's database engine from MyISAM to InnoDB SO THAT the table will have better performance with locking

Note: I'm not a DBA, but I play one on TV. What I'm about to explain I have not tried in a production environment. So I don't really know how much performance will be gained by this. Your actual mileage may vary.

One of the engines that MySQL uses is MyISAM. As I understand it, that engine uses table-level locking for all updates. This means that if an INSERT or UPDATE is happening to a table then any SELECT on that table will have to just get in line and wait for the INSERT or UPDATE to complete. Usually things happen so quickly that this is fine for most applications. If you run the command SHOW FULL PROCESSLIST and see a lot of processes in a LOCKED state (those are the ones standing in line) you might benefit from switching to the engine to InnoDB for a few of those tables.

So my plan is to create a new InnoDB-engine table which looks exactly like my existing MyISAM-engine table. The example I'm going to use is the "users" table:

Step 1. Generate the CREATE script for the existing table.

SHOW CREATE TABLE USERS;

Step 2. Copy that table creation command, change the table name (I just append '_innodb') and change the engine from MyISAM to InnoDB, and execute the command.

CREATE TABLE `users_innodb` (
 `uid` int(10) unsigned NOT NULL auto_increment,
 `name` varchar(60) NOT NULL default '',
 `pass` varchar(32) NOT NULL default '',
 /* and other columns */ 
) ENGINE=InnoDB AUTO_INCREMENT=10 DEFAULT CHARSET=utf8

Step 3. Copy all the rows from old MyISAM table into the new InnoDB table. (Note: make sure nobody else is modifying the database while you're doing this).

INSERT INTO users_innodb (SELECT * FROM users);

Step 4. Swap table names around. Rename the tables so your application will start using the new InnoDB table you just created and filled.

ALTER TABLE users RENAME TO users_myisam;
ALTER TABLE users_innodb RENAME TO users;

That's it! Goal accomplished. You're now using InnoDB as the engine on that table. Repeat the process for any other tables you feel could benefit from InnoDB's improved locking performance.

Slow File Chooser in IntelliJ and NetBeans (actually it was a Java JFileChooser bug)

noreply@blogger.com (Brad Parks) — Fri, 01 Jan 2010 05:26:00 +0000

I use Eclipse for just about all my Java development and Flex Builder is also Eclipse-based, so I've never really had to venture beyond Eclipse for my development needs. But I decided to expand my world to IntelliJ IDEA just to see how it stacks up.

Perhaps more on that later.

After installing it on my Windows XP laptop one of the first things I noticed was that it took a painfully long time for it to show a file browser dialog. Painful. All I wanted to do is save or open a file. That's not too much to ask. My first thought was that there's no way I could use this for my daily development. (Don't stop reading yet; the important part is forthcoming.) Saving an open file was fine, it was just the file browser that was problematic (for the Open... or Save As... operations).

So I downloaded NetBeans to see what that was like. I experienced the same problem. I tried them out on my Windows XP desktop machine and they worked just fine. Y'know, come to think of it, when I use the Java S3 file browser jetS3t the file chooser is also slow. Similarity identified -- it's the Java file chooser on this laptop.

Quick check of the Java bugs database found quite a few related to JFileChooser being slow on XP that had been fixed.
Quick check of the Java version installed on the laptop: JDK 1.6.0_06. Yep, I'm a few update revisions behind.

So now I've updated to JDK 1.6.0_17 and all is well.

IntelliJ IDEA is fast and cool. NetBeans is fast and cool. Both of them will take me a while to get used to since I'm fluent in Eclipse and of course shortcuts and features are just different.

hibernate.connection.url for Various databases

noreply@blogger.com (Brad Parks) — Thu, 31 Dec 2009 15:28:00 +0000

hibernate.cfg.xml -- Here are some example hibernate.connection.url settings for various databases:

Oracle


      
          jdbc:oracle:thin:@hostname:1521:nameofsid

SQL Server


      
          jdbc:sqlserver://hostname:1433;databaseName=WHATEVER

Hypersonic


      
          jdbc:hsqldb:hsql://hostname