Day 62

At last I understand. For quite some time now, I have been doing a lot of work on websites and web apps. Working on a variety of different frameworks, on several different projects. But I have always been working on the back end. Creating models, creating functionality, making interactive pages, and migrating data; Yet I never did any real work on the front end. I suppose my fantastical UI design for my TicTacToe app counts, and maybe the infinite beauty of my Crown Imports Mockup with Balsamiq, but no real front end web pages.

Today I at last learned how it is usually done. I also learned how bad I suck with photoshop, but thats another story.

Basically, you will hire some designer (perhaps even you can do it!) who knows a lot about making pretty things and making things that make sense for people who like things that make sense. This dude will whip up some totally awesome image of your webpage using photoshop or its equivalent. Then, after he has perfected every corner and nick nack, he will ship it over to you.

Hopefully, it is nicely layered and can be easily sliced, but even if its not you can still get the job done.

First, you need to decide which elements are going to be important and static. For example, most web pages have a fixed header and footer. You need to identify such items and extract them into their own images. Make sure you only grab parts that you know are going to be the same on pretty much every page.

Next, you find all the semi dynamic elements, like buttons and check boxes and such. There are some default buttons for most browsers, but you might want special buttons that look super cool and sparkle when you press them. Find these buttons and make sure you get both the default image, and the pressed image. You probably always want a mouse over image, so don't let your designer get away easy!

Once you have these elements, extract them out into their own images, keeping in mind they will be popping up all over the place.

Then, you spot out all the really dynamic items that are likely to change a lot. Animations, rotating picture frames, and text boxes/ text labels. All of these items will of course require different attention, but a few key things to keep in mind are make sure the text sizes the designer used will work in the browser and in your elements, make sure can include png images (Drew Diller has some useful tools for this), and make sure you rasterize any image you want to be transparent.

All these super dynamic items are going to require special attention, and there really isn't just one way good way (at least that I have come across) to handle them.

Finally, all these images and elements need to be shoved into some html so that they can be displayed on your web page. From what I have seen, some people like to use html tables to organize all the items, where as other people prefer using offsets to line everything up they way they like. You can do this in either html or css, or both, but css is a nice way to organize all the display information while also giving a lot of extra benefits.

Thats the general process of setting up the front end of a web page. After that you can add in some nice JavaScript to give you all the dynamic behavior you need, but make sure you are writing tests for that JS!

If you have any improvements or suggestions, or even good articles you know of that give further detail on the process, please let me know!

Day 61

So, I was going to do a blog on how I set up my testing framework for JsUnit, and made a little interface to use it and all... but today I discovered ScrewUnit! After reading Corey Haines' blog on his stickies project , which was very much about testing JavaScript and all these neat ways to do TDD with it, I am now a complete convert. It is sooooooo much better than JsUnit. The error messages actually give line numbers! How weird is that?

BlueRidge and ScrewUnit are largely for Rails, but of course ScrewUnit can be used for just JS. It is an excellent framework, built up almost like RSpec, but of course the syntax is different. It is run in the browser, and you can just click on a test, or a describe block and it will run it. Very fantastical.

Since everything in JS is a object, and you can pass functions back and forth, most of the time mocking out objects is trivial. In fact, when you design your app using TDD, you can make it soooo versatile that you can just pass in functions or blocks that act as natural parameters, but can also do your test for you. Hopefully once I start my next project I will have some nice examples to show this in action.

I also learned a lot of JQuery today. How weird is it that you can get JQuery in one 6,000 line file? There are actually lots of JQuery libraries, but I just yoinked the recommended file for now. Pretty neat tool though. It will be nice not having to do a document.getElementById("") all the time, as well as making it easy to add classes to elements for easy styling. Something that will be exceedingly useful in the project I am about to start.

Speaking of which. I just estimated a new project, which is supposed to be a web app? The final nature of the program is still a bit of a mystery to me, which certainly causes issue with estimating, but it is supposed to be done nearly entirely with JavaScript. It is also a fairly large application. Not super complicated, it just has a lot of pages. This is why I wanted to learn a lot about ScrewUnit and JQuery, so that when I started working on this project I wouldn't do it page by page, but I would have the ability to create templates. It could take ages to finish if I had to design each page anew (there are actually a huge variety of them), but if I can get some really useful objects, then I am pretty sure I can start cruising!

Also, my classes are starting to become a pain now. Which is unfortunate. I have somewhat neglected them recently due to my apprenticeship challenges, and now it is coming back to haunt me!

8th Light Blog Post: JavaScriptness.prototype = new Class(); From Classical to Prototypal

JavaScript is a very misunderstood language.

It has gotten a bad rap because most people that use it don't look for its brighter side. Instead they will often look online for some code to copy paste, or they will hack something together using tiny snippets and in-lines to get a drop down menu.

It is also a very conflicted language. By nature, JavaScript is a Prototypal language where nearly everything is an Object and there are no classes. It is also somewhat Functional since all the functions are First Class Functions. They are just more objects that can be passed into and returned from other functions.

The issue is that many people try to use JavaScript like they would use Java or C++. They want a Classical language, where they strictly define all their types and then create instances of those types. Even though JavaScript can be jammed, beaten, or even mutilated into Classical structures (in fact the book JavaScript: The Definitive Guide spent a chapter on exactly that), this is betraying its nature and ignoring all the benefits that come with Prototypes.

The Classical Form

Lets take a look at a basic Classical implementation of a Square using JavaScript:

function Square (side) {
  this.side = side;
}
Square.prototype.area = function () {
  return this.side * this.side;
};
Square.prototype.perimeter = function () {
  return this.side * 4;
};

var mySquare = new Square (5);

You will notice that JavaScript even has a new key word in an attempt to match a Classical implementation. The function Square () defines a class with a side attribute. It also has two instance methods, area and perimeter. We can then create mySquare in almost the same way we might do it in Java. This is JavaScript though, and the new works differently in JS. It creates a new, empty object and then calls the Square() function (or the constructor) passing the empty object into this . The constructor then populates that object with the specified behavior before returning it. This is particularly scary, because if you forget to put new in front of the constructor, this will refer to the Global Object, as will mySquare (meaning any further changes made to mySquare would also be on the Square 'class'). This is why, if you want to use JS in the Classical way, you should always capitalize the first letter of a constructor.

Ok, so we have our Square, but say we want to create a square that holds a X or an O. Well, we would want to inherit the properties of our current Square 'Class' and then add to them. We might do something like this:

function ContainerSquare (side, contents) {
  this.superclass(side);
  this.contents = contents;
}

ContainerSquare.prototype = new Square();
ContainerSquare.prototype.superclass = Square;
ContainerSquare.prototype.constructor = ContainerSquare;
ContainerSquare.prototype.getContents = function () {
  return this.contents;
};

var myContainer = new ContainerSquare(6, "X");

We create a new 'Class' for the ContainerSquare which uses the Square's constructor to define the side attribute, along with the two instance methods. We then define our new contents attribute, along with an accessor method to go with it.

The ContainerSquare .prototype = new Square();
line sets up a Prototypal inheritance structure so that a ContainerSquare object will be linked to the proper prototype chain.

The prototype chain is the hierarchy of objects that a method or attribute call will traverse until it finds what it is looking for. In other words, when I call myContainer.size(), JavaScript will first check to see if the myContainer object has that function. If not, it will proceed to check the object's prototype, which is the ContainerSquare. Since ContainerSquare doesn't have the function, next in line is ContainerSquare's prototype, which is just Square. Square does in fact have a size() function, and since javascript will use the first function it finds, it will use Square's size() function.

Thus, by setting ContainerSquare's prototype to Square, we inherit any of the attributes or methods of Square.

Although this all works, and we have successfully mashed a Prototypal language into a Classical scheme, we should try this in the way JavaScript wants to be used.

The Prototypal Form

We can start with the Square. There are few pretty simple ways to achieve the same functionality using regular objects and their prototypes, and we will look at two of them. The first will create a square object that we can use. Then if we need more squares, we can make a copy of that object using its prototype. The second will create a squareMaker function, which can then be used to pop out new squares.

var firstSquare = {
  side: 5,
  area: function () {
    return this.side * this.side;
  },
  perimeter: function () {
    return this.side * 4;
  }
};

Here, we already have a new and usable square at our disposal. We don't need to call a constructor and define any types. We can just take an object and mold it into the form we want.

Say we want another square though. We can't just write something like var secondSquare = firstSquare; because JS passes objects by reference, and thus the second square would just point to the first square. What we can do, and this is a technique developed by Douglas Crockford, is make a copy of our object by calling a new constructor with a prototype that points to our object. This new contructor creates a new empty object and assigns the values of our old object to the new empty object. You can do this yourself, or you can use Crockford's technique as follows:

if (typeof Object.beget !== 'function') {
  Object.beget = function (o) {
    var F = function () {};
    F.prototype = o;
    return new F();
  }
}

var secondSquare = Object.beget(firstSquare);

secondSquare.side = 6;

You can see that we are actually defining an on the fly constructor F, using our object 'o' to define F's prototype (where 'o' is firstSquare). Then we create a new object using that constructor. This will give us a new copy of our object, with all its attributes and functions. If we now called secondSquare.area(); we would get 36.

If you wanted to then make a ContainerSquare, you could simply add a contents attribute to the second square, and then make copies of the secondSquare if you need more Containers. Keep in mind that since firstSquare is the prototype of secondSquare, if you were to add a contents attribute to firstSquare, you would then have that attribute on secondSquare; however, adding attributes to the secondSquare does not place them on the firstSquare.

Spawn More Protolords

The other way to get squares would be to make a squareMaker function. This function will return a new object with whatever attributes you define. I am also going to show you some closure so that the attributes are private, and only accessible through accessor methods.

var squareMaker = function(side) {
  return {
    getSide: function() {
      return side;
    },
    area: function () {
      return side * side;
    },
    perimeter: function () {
      return side * 4;
    }
  };
};

var anotherSquare = squareMaker(5);

You will notice that the return value of the squareMaker is almost exactly like how we defined our firstSquare object. We are just returning the definition of a square, and thats quite awesome. Another thing you will notice is the bit of closure. We pass a value into the squareMaker, but it isn't stored anywhere. It is held in the scope of the squareMaker function, allowing those internal methods to use it, but hiding it from the outside (unlike the side attribute defined in the firstSquare). This sort of closure is another wicked awesome tool you can use in JS. In a way, this use of closure is like defining private variables to a class.

To get the Container functionality we create a containerMaker using the squareMaker, and some more closure, to make a new square object. We then dynamically add a getContents method to provide access to our private contents attribute.

var containerMaker = function(side, contents) {
  var container = squareMaker(side);
  container.getContents = function () {
    return contents;
  };
  return container;
};

var anotherContainer = containerMaker(6, "O");

Conclusion

So we have now seen two ways to use some of the better parts of JavaScript to get the same functionality that we could get using the Classical scheme. We can also see some advantages in using JS in its natural form like: getting some closure (which can be exceedingly powerful), very dynamic objects ready to change in anyway you can think of, quickly defined objects which can be used and multiplied, and no need to predefine types. Pretty sweet right?

I will admit one slight inefficiency with the two solutions I showed you (although there are ways around this). Using the squareMaker, or making copies of the objects will make full copies, including the function objects defined inside. If you use the Classical scheme I showed, you are defining functions on the prototype rather than the object itself, thus there will only be one copy of the function.
You can, of course, do something similar in the JS scheme, but the optimization doesn't count for all that much in most cases (unless you are doing mobile development where you want to save everything you can!).

JavaScript isn't a Classical language, but instead a Prototypal language that is powerful enough to mimic a Classical language without breaking a sweat. There is much to be gained by recognizing this fact and changing your mindset to work with it. Trying to fit a square into a circular hole will just get you stuck. All languages are unique with a variety of their own advantages, and as good developers we should recognize these differences and reap all the benefits they have to offer.

Day 59

Java Script!! So I finished my unbeatable TicTacToe last night... or this morning really. For only having 4 days (2 of which I was still quite sick) to learn a new language (2 really, since I didn't know HTML very well at all either), to learn and customize a new testing framework, and to create a new TicTacToe program, ai, and interface, it turned out really awesome. The interface was a little spartan, but it was fully functional with bonus features.

Though this was probably just because of the tools I was using, I found that I had to be exceedingly careful with Java Script. There are a lot of little things you can do or forget to do that will really mess you up. Since the code is all compiled in the browser, and most things in the language are done at run time, the errors can be pretty tough to chase. Even when using a testing framework like JsUnit, you just don't get very much valuable feedback from error messages.

As a result, I worked extremely carefully, doing everything slowly and very incrementally. I never made a huge changes without running my tests again and again, and making sure everything still worked along the way. This way, most of the errors I had were found within just a few moment by scanning through the recently changed code. I was also fortunate enough to have all the basic rules of what to do and what not to do fresh in my mind from the Java Script: The Good Parts book.

Another thing that I did, that I know saved me in a huge number of cases, is I would very, very frequently stop moving forward and move back two steps. Rather than adding another test and getting a new feature done, I would look back at what I had and see if I could find any better implementations. At first I did this just to try to make use of some of the cool benefits of the Prototypal and Functional nature of JavaScript, but after awhile it just didn't make sense not to go back and refactor. I found time and time again that if I hadn't turned back to refactor when I did, if I had just delayed it for one or two more features, I would have been stuck with a mess. At that point, I probably would have just kept what I had and continued to develop a messier and messier program.

It was also really easy. It just seemed like Java Script is the type of language that wants to be refactored. Perhaps its because I wasn't used to it. But it is also possible, because the language is so straight forward, so direct, and so expressive, that there is almost always a obvious and significant improvement that you can make.

Testing JS was a bit of a pain at first. Actually, it was a bit of a pain most of the time. Once again, this may be from my lack of experience, but JS and HTML seemed so intertwined that decoupling the logic from the GUI was often a challenge.

For example, to include a JavaScript file from another JavaScript file you have to actually generate some HTML to include it! I didn't want to explicitly write out

<script type="text/javascript" src="another.js"></script>

right in my first JS file, so instead I added another function to the document object which creates a script element, and plugs in the file you want. This way, anytime I wanted to include another .js file, I could call that function in my current .js file, and avoid having to write out the script line (which btw is fickle way to do it because certain browsers (like IE7) will read your <> ... < / script > as the tag for the previous include, and thus will mess everything up. So you actually would have to do <> ... < / scr + ipt > so that the tags couldn't be read until they were concatenated by the parser, and thus safe).

Here is how I did it:

document.addAnotherJavaScriptFile = function (filePath) {
 var head = document.getElementsByTagName("head")[0];
 var newScript = document.createElement('script');
 newScript.setAttribute('type', 'text/javascript');
 newScript.setAttribute('src', filePath);
 head.appendChild(newScript);
};

document.addAnotherJavaScriptFile("game/computer.js");

This made it easy to include more files, if need be, and was a guaranteed way to work across browsers.

I am completely exhausted now and am gonna go pass out. Something I discovered last night/morning - when you are at the point where your brain shuts off (maybe around 2:30 - 3 AM) but you still have to get things done... it might honestly help more to just start trying random things and use guess and check rather than thinking it through. After I spent about 45 -60 minutes trying to work through a problem step by step, with no success, I decided to just start switching the important variables in a nearly random manner with the key values, and within about 3 minutes I stumbled upon the solution. Of course in the morning I spotted in almost immediately, but desperate times often require desperate measures. And hot damn, it got the job done.

Tomorrow I will go over some of the ways I used JsUnit to test my code. I will also be handing in my 8thLight Blog submission tomorrow, so that should be interesting.

Day 57

So apparently driving for 24 hours straight can get you sick. I got back from my vacation to Florida, where I visited Panama City Beach (Amazing!), and Miami for the Ultra Music Festival, and the drive back was a grand total of about 1,400 miles and 24 hours of driving. Since we wanted to get back on Monday at a decent time, we drove straight through the night with scattered chunks of restless sleep as me and Tomato switched off. 5 monster energy drinks and a bunch of Oreos later I was home.

Unfortunately I must have caught something in Florida, and the lack of sleep mixed with a lack of nutrition and the overall unsanitary environment which is my car, my immune system was crashed.

Tuesday I felt under the weather, but was still able to get a lot of reading done. Colin, I finished the Java Script book, which was excellent by the way, and I can get it to you tomorrow. If you don't mind, I might want to keep it for reference though because I am going to do my 8th Light blog post on turning a Classical Object Oriented/Class driven batch of code into the true Prototypal nature which is Java Script.

Wednesday I was in bed for nearly the whole day, my body completely stripped of energy and trying to over come whatever it was I had/have. A soar throat, relentless coughing, wavering headaches, and a complete lack of energy left me almost completely worthless. I did still manage to mess around with getting some basic JavaScript and HTML interaction going. I was pleased to find out that it is actually exceedingly easy, but that was after I angrily grouched about trying to find out why certain things weren't working only to find out that there was a typo in the book! A simple little mistake, very illusive and sneaky. The book was defining a literal with one attribute and one function like so:

var myObject = {
        value: 0;
        increment: function (inc) {
                this.value += typeof inc === 'number' ? inc : 1;
        }
};

myObject.increment();
document.writeln(myObject.value); // 1

Can you spot it? The actual code chunk was a little bit larger, but this is where the error lies.

Having no real JScript experience, I was completely baffled why my very first test program (after some other simpler stuff like 'Hello World' and such) wasn't working.

See it yet?

Its the semicolon after the         value: 0;

It is supposed to be a comma... Blast you editors not catching publish mistakes!!

When defining attributes of a literal you use the format
{ name: value,
name2: value2,
name2: value3
}

Oh well...

Today, although I still felt kinda crappy, I was able to get a lot done. After playing around a bit longer, getting a better feel for the relationship between JavaScript and HTML, I spiked out a complete TicTacToe interface which was totally playable and such, but no AI. Now I am working on the real interface using TDD and JSUnit. JSUnit is a lot like JUnit, but I am making my own test page thingy, which is pretty sweet.

I have a LOT to do yet, so tomorrow and this weekend are going to be packed tight!

Day 54

I received my next two apprenticeship challenges today, and I am quite psyched to get them done! The first is to write a more formal and informational blog post for the 8th Light website. The second is to write an unbeatable Tic Tac Toe in a language I have never used before.

For the blog post I was thinking I would write a tutorial on how to get started on a new Limelight app, but it turns out Paul Pagel already wrote one of those. It might still be a good idea since Limelight has evolved quite a bit since that blog, but I am not sure yet.

As for the Tic Tac Toe in a new language, I was thinking I would do it in java script. Not my first choice in languages, but I will have to learn JS soon anyway to do a project Paul has been asking me to work on, which will be pretty much entirely done in JS, as well as a little html and css. I kind of want to do it in a functional language like clojure since I have almost no experience working in that frame of mind, but JS would be a twofer.

Here is the catch though. I have the next two weeks, plus a weekend, to finish these two tasks, and I will be gone in Florida for the rest of this week! That means I have only 7 days to get these two tasks done... I see some rough nights in my near future, and I am damn excited to make the most o them!

Today I did some more work on Limelight, getting some more tests to pass and fixing up the margins on the text area.

I also got royally pranked by two esteemed members of the 8th Light team. I will let them remain unnamed for their treachery, as no good prank is left unanswered.
Basically, I had gone to the DMV today to renew my license, but of course it turns out they are closed on Mondays. Either way, I mentioned this at lunch in the office. Later on, I left my iphone in the bathroom where it was recovered by one of the Craftsman to remain unnamed. Rather than return it to this unsuspecting, trusting, and apparently oblivious apprentice, this Craftsman, along with another Craftsman, decided to see what fun they can have.

At about 3 pm I get an email from a Jacob Carson, who says I left my ipone near the Libertyville DMV and they wish to claim the reward (I keep my info and a $50 reward offer as my phones default background). This completely blows my mind as I specifically remember walking into the office listening to a podcast when I returned from the DMV. Suddenly I start frantically retracing my steps in my mind trying to find out how this could be possible. Did I hallucinate having my phone when I came back? Could I have been listening to my nano? Was I thinking of a different day?

I eliminated all these possibilities, and it seemed to me that there remained only one possibility. Neither Micah nor I use an iPhone case, and thus they appear remarkably similar face down. I conceived that it was possible I had left mine in the bathroom, which Micah must have later gone into and mistakenly taken my phone for his, at which point he went out for lunch (with a client) at the subway right next to the DMV. There my phone must have somehow slipped out of his pocket, thus returning it to the place I had been earlier today. An outrageously unlikely possibility, but Eric Smith wasn't in the office that day, so it must be the case.

I went down and asked Micah if he had eaten at subway today, and of course he said no.

Perplexed... I proceeded on to the bathroom, to the origin of the problem. Once I step out, there lies my phone on the ground before me.

It was so obvious, but so very devious. Touche good sirs, touche.

Day 53

The presentation went quite well, and I got a lot of excellent feedback. I was also able to talk to my dad and get some more ideas.
Here is what I am going to do:

First I will lay out a hypothesis on what good code looks like. I will then choose a batch of code snippets that represent my hypothesis, and I will also select some snippets that are nothing like my hypothesis, as well as some random snippets.

Next I will collect audience feedback on the snippets and use it to refine my hypothesis. I will take their suggestions to refine the snippets and further polarize them.

After I implement the last group's suggestions, I will show the snippets to a new group and see if there has been any improvement. From this I can begin to derive some knowledge about what people 'think' good code looks like, and what people will actually rate as good code.

This discrepancy alone will be valuable because it will demonstrate whether or not we know how to write code that we will like. As most developer's know, their own code is always excellent and clever... for a few months, but when you come back to it a year later it might not be so good. This could be because we have gotten much better at writing code, but it could also be because we never really thought it was good in the first place. We merely implemented what we thought were good practices, and not what we actually valued in code.

Once I have gathered enough data, and refined my methods such that I can accurately predict if people will perceive a code segment as good or bad, I can then work to create a flurry of good code segments.

With these I can begin offering more talks, still collecting data, but also performing a little Clockwork Orange. I will be able to show slides with code that I know almost everyone agrees to be good, and use them to 'condition' the audience.
I can perform little experiments where everyone writes a small function at the start of the talk, and then every 5 slides people will pass their functions around and have others rate them. We will see if the average ratings change, and for what reasons. We would then write another function half way through, perform the same task, but see if the ratings have improved.

Lots of value to be gained here, and I don't even know the half of it.

Day 52

Tomorrow I am giving my Lunch and Learn talk! I am actually pretty excited for it, and to begin my research process. There is a lot of knowledge and experience out there, and I want to go grab it all! I got that nice keynote remote app for my iphone, it really helps keep things going smoothly. There is also the option to have speaker notes on your phone, so if you get caught up at any point you can peak down at your remote and get back on track.

Since this is the first, of hopefully many future talks on the Code Sense, it will be a little more on the experimental side. I have created a formula which I think will allow me to gather a lot of important information, and even stir up some debate, but it is still a very fluid concept. I am hoping to get a lot of feedback tomorrow, hear what the guys think, and drastically refine my process to be prepared for the conferences to come.

I apologize for not being more on top of my blogs recently, work at the client site has been rather straight forward recently. Although the feature Micah and I are working on right now is posing some challenges. I actually spent most of the day turning a very complicated cucumber step into a valuable test. The acceptance test was very stand alone, and there were none of its kind before so everything has to been done nearly from scratch. It is also fairly monstrous because, although its purpose is to test only our new feature, its set up is ridiculously complicated.

At first, and this is an issue that really became apparent today, I was going to make all the step definitions to walk through the tasks needed to set up the exact circumstances we were testing. However, this was just catering to our solution. I was going to set up the circumstances in the database to exactly match what we check for. But all this really checks is that our solution does what we know our solution does. This is of course what most unit tests do, but it doesn't work quite the same way with acceptance tests. Thats why there are nice helpers like Webrat that will set up the database by going through the actual steps a user would go through. Thus, I began reworking all the steps to set up our scenario the way it would really be set up in production.

This was much, much more complicated. Creating the scenario to test what we wrote, going the full route, meant that our test had to touch an enormous portion of the code, and exercise a huge conglomeration of classes. I honestly think this is going to be one of the longest, if not the longest, of all the client's acceptance tests. Maybe it should be broken down, or maybe I am missing something, but this is what is required to recreate the circumstances we need to really test the production code.

Day 50

Micah and I applied to speak at Rails Conf in order to begin my research. I hope we get accepted as Rails Conf is a huge conference and would be an excellent place to get my name and idea out there. There is also another conference coming up in Chicago where I will submit my proposal. This is a smaller conference, but it will have a lot of the Chicagoland software companies like Obtiva and 8th Light. Whether or not my proposals get accepted, I am excited to begin diving into this research. I thought out most of my talk for Friday, and I will begin creating the slides tomorrow morning.

At work today I began chugging through our new feature. It was quite a pain in the arse because I basically spent all my time trying to decode massive conditional statements. The code is trying to use a single query on the database to filter out a massive number of objects with very specific requirements. As a result, there are these overwhelmingly complicated conditional statements that are nested in nests of nests inside a nest cave hidden in a gap in another nest. Not only are the many levels deep, they also mix together ruby methods, railsy methodness, sql queries, and ruby methods that perform several sql queries and. Some of them are also inefficient, and could be logically reduced and simplified. I would do this, but I don't know that the algorithms have a comprehensive test set. I would make one, but I have to make my own batch of tests to finish the feature, and if I stopped what I was doing to write tests for every chunk of untested code I found, I would never get a feature done.

Once again, if you have any suggestions for my talk, I would appreciate the input.

Day 49

Today I was charged with my first of three final challenges for my apprenticeship. The first one, which needs to be ready by this Friday, will be to deliver a lunch and learn to the 8th Light group. Though I already gave a lunch and learn on my testing design patterns, this lunch and learn will have to be pristine. I already know what I will be doing, and I will explain this further in a moment.

At the client site today Micah and I got to see the fruits of our labor. Our feature is not only out on the live site, but is now in full use. The feature was the ability to have, make, and use promotions in order to acquire more members. Today they sent out a massive batch of emails with a promotion code, and in just one day, the promotion dragged in over 2500 new users. They were quite pleased to say the least.

We also finished up another feature which sends out reminder emails to people who have subscribed using the promotion. We then picked up another story which we will work on tomorrow.


My plan for my first Apprenticeship Challenge:
This is actually something I had started talking about with Micah on Thursday last week, and we discussed it even more today. I had the idea when Micah was explaining to me how his talk went at the conference he recently spoke at. One of the activities he did was to show his audience a bunch of code snippets (this was originally done by Michael Feathers, and he actually gave the snippets to Micah) and have them rate each snippet on the quality of the code. After everyone voted on a snippet they compared everyone answers only to find that everyone had different opinions on which snippets were good and which were bad.

This got me thinking. Good code is a very subjective concept. There are certain things that people will agree help the code, or hurt the code, but for the most part... there is no one formula to write good code. In fact, great code to one person can be merely decent code to another. Likewise terrible code to one developer could look very readable and understandable to another. Such is the debacle. What is good code?

My dad recently wrote his Clean Code book which explained certain characteristics that help to define a process that could lead to good code, but at the same time it was clear that it takes years and years of practice to learn how to develop consistently good, clean code.

But does it have to? Maybe there are certain aspects of code that all people find pleasing and understandable. Maybe there exists certain naming conventions, or certain malleable themes that really help to explain intention. I don't know, but I am going to find out.

My plan is to go to conference after conference holding the same talk/workshop where I hold activities showing people code snippets and getting their feed back.

Here is how I, and this process is open to all suggestions, plan on doing this.

First, I need a way to get people to come to my talks. I need an incentive. Well, how about I offer expert advice on what 'Good Code' looks like! Damn... I am not qualified. Thank goodness I know a lot of people that are! I will collect a wide sample of code snippets and then ask several industry experts which snippets they think are good, and which they think are bad. Then I will offer to show this information at the end of my talk, after we see what my audience thought about the code.

Once I get people at my talks, we will proceed to look at a lot of code! In some fun manner I am yet to create, I will show the audience a code snippet and then ask the following questions:
How is this code? (possible answers = Awful[0] - Bad[1] - Ok[2] - Good[3] - Excellent[4])
(They have to answer this, and keep their answer no matter how they answer the rest of the questions. The point is to capture their 'Blink' estimation, or their first impression of the code)

How is the naming?
How is the use of the language?
How clear are the levels of abstraction?
How complicated?
How consistent and useful is the theme?


After we have rated a batch of maybe 5 snippets, we will go back an examine them. I am not to sure about this part because I want to keep the review until the end of the talk so I get mostly unbiased data, but I don't want people to get bored.

Finally when we are all done we will compare what we thought about the code to what Industry Experts thought of the code.

Throughout the talk I will be asking people for their thoughts on the code, on their reaction to everyone else's opinions, and especially for reactions of the expert's opinions of the code.


8th Light will be my first group and first experiment. This will be the trial run to work out the kinks in my future talks and presentations.

Let me know what you think and please offer suggestions!!!

Day 47

Another successful day as we were able to finish all of our features for this iteration and commit everything in time for deployment. I have really just been learning a lot about how to become a part of a team. This has really been my first time doing stand-ups everyday, sharing and hearing about all the progress made and what is yet to be done. I am now very confident in the stand-ups, and I actually listen to what everyone else has been doing rather than spending all that time preparing what I am going to say! I can interact with the customer, handling their requests and really digging deeper into the requirements of a feature. Not only the technical requirements, but also what the real business requirements are, and the purpose behind every aspect of a feature. These are most definitely crucial skills in becoming a Craftsman.

Cruising along in the Rails book, although haven't been able to make too much progress the last few days as I was busy with the new features for work, and then school work yesterday and today. Unfortunately I have a test tomorrow, though I might take it on Saturday, so I have to spend some time catching up on that :/

Micah and I were having some very interesting conversations today, ranging from massive databases spontaneously becoming self-aware to the foundation of all forms of reason.

Can a sufficiently sized database suddenly become aware of its existence? It would be difficult to back any conclusions on the subject with concrete facts, given that we have no real examples of this happening. Or do we? I am tempted to say we do. Animals over time developed larger and larger brains. Now a huge brain on a huge animal has to use most of its real-estate on muscular control, but when you have a smaller animal with a large brain, suddenly you have a lot of extra space. With all this extra space, the brain will just begin to collect data. Each experience will create new connections between neurons, and thus a new route to the same destination. These connections are just new associations with different contexts. Once you gather up enough associations or connections between common objects and experiences, you become aware of them. This is likely how humans became the sentient self aware creatures that we are. Our minds became sufficiently advanced and could collect a sufficient quantity of knowledge such that we became aware of certain objects that we could manipulate. Over time this advanced into practical knowledge, and with the creation of forms of language people could share such knowledge. Then of course writing drastically accelerated this, since the conclusions of new generations no longer had to be gathered independently or passed down through legends and tails. But all the advancements of our minds came through more associations with new and old ideas.

Admittedly, I am no neural scientist, but I believe the basic principle behind the function of a human brain and the function of an AI is quite similar. When I child is trying to figure out the world, how do you teach them what an apple is? You can't just start saying "Apple" and expect them to understand, unless you give them a context. You show them an apple, point to it, and say "Apple". Suddenly an association is formed in their brain between the image of an apple and this strange sound. Do this a few times and they will recognize it because their brain will have made enough associations between the separate concepts. Such is similar with an AI. Take the word "apple" and connect it with 1,000,000 pictures of different apples, give it the ability to discern certain features, and suddenly it can recognize apples (sorta).

Escalate this. Take it further and further. There is already hordes of recognition software out there, as well as images in association with apples. Naturally there is a huge leap from being able to recognize an apple and becoming self aware, but I would purport that it would be possible, given enough processing power, for a massive database to become self aware. Certainly not always, but it doesn't seem too far out of the question. Especially with tons of science fiction novels explaining the entire process like manuals.

Just some thoughts of a wandering mind.

Day 45

Today I was left to fend for myself. Once again, Micah was gone, and I was in Chicago representing 8th Light. Unlike yesterday, however, where I was mostly left to do my work, today I was really sucked into business. This is a fast moving company, constantly adding new features and new code, and thus any issue that arrises needs to be taken care of very quickly. Since Micah was gone, it naturally fell that all questions, in relation to what Micah and I had done recently, came to me. It started with a quick question, from the CTO, about our design on one of the features. But that rapidly turned into several more questions on if it worked how it should, what the requirements were, and so forth. Eventually this resulted in a new feature that needed to be added on top of our old one.

This started it all. See once the precedence had been set, and people saw that I was standing in for Micah, suddenly I was Micah! Within just a few minutes, I had become the new 8th Light guy, and all the questions and information that had been going through Micah started coming through me. Honestly about half the time, when they were telling me all the changes they were making and asking me to get something done, I didn't know what I was supposed to do. But I took down notes of what they were saying, like I had seen Micah do, and then when I had the time I went and learned what it was they were talking about and what I had to do. I called Micah several times to try and keep him in the loop, and fortunately not a whole lot had to get done immediately, but there was a lot more going on compared to Monday.

By around noon, I had gotten a hang of it. I felt comfortable answering questions on what we had done, discussing the things that had to be done, and taking responsibility for the work 8th Light was doing. It was actually pretty awesome. From yesterday to today my role changed from someone who was just working in the office to someone who was working with the office.

Unfortunately I was still having some trouble working out the details of how to test my feature. So around two, I decided to take a break from that feature (to wait to work on it with Micah), and start the new one that was created today. This feature came along quite smoothly and I have nearly finished it, hopefully tomorrow on the train I can claim it as complete.

It was fun because even though I didn't know exactly how to do it, I knew how to find out, so I ended up learning a lot along with getting to experiment a bit with one of the other guys there on how to do it. Mostly we just played with the placement of error messages, how they should be formed and where they belong. Stuff like, whether is should be a full message, or use Rails built in concatenation stuff. Whether we should stick the messages in the controller or the model, and if inside the model, should it be attached to the model or pushed up higher. For the most part, it was a lot of quibble matters, but still helpful for me to get a better feel for their program.

Micah will be back tomorrow, but now I am ready to get a lot more involved on the business end of things.

Day 44

I need to make this quick since I have to get up quite early. Commuting really makes you feel how short a day can really be. When you get up early, and suddenly its time to sleep, you really notice those 3-4 hours lost everyday in transit.
I've made a fair amount more progress in the Rails book. I really enjoy how this book teaches you how to make a full project, start to finish, as well as giving you extra side tasks to get on your own. It seems like this is the most effective way of learning, and is beginning to make me critical of all those classroom lectures, and tedious homework assignments which took your step by step through the learning process without having you actually do anything memorable. I think that a class structured around a single big project, coupled with many side tasks and activities to help accomplish the main task would be far more effective than flurries of small side projects that you forget the moment you hand them in.

Tomorrow I should be finishing up my task at the client site. Micah is gone on a conference so I am in Chicago working alone with the client. My biggest problem is encountering mental blocks, where some little missing detail prevents me from putting the full puzzle together. At the moment, this mental block is stopping me from both testing, and seeing how the next step will work. Hopefully tomorrow morning I will have time enough to collect that detail and fit it into place.

Day 43

At 8th Light, every once in awhile they will have some internal reviews. It is a way for everyone to learn all they ways they can improve, and to make educated goals to work towards. I sat in on these reviews, taking notes, and looking for common trends and things that were frequently suggested, or traits that were frequently praised.

I came up with a batch of traits and qualities that seem to highlight many of the virtues you would find in a Craftsman, as described by 8th Light. I thought it would be valuable to have such a list so that I don't have to wait for a review to begin setting my own goals!

*Strong Enthusiasm - Always brightening everyone's mood with your love of what you do.

*Don't be afraid to ask for help - Especially if you feel rushed, or have bitten off more than you can chew.

*Ask a lot of questions - Even if you are already a Craftsman, if you are surrounded by other Craftsman, there will always be new things to learn.

* Discipline Discipline Discipline - The mark of a Craftsman is to have an unfailing discipline to do the right thing. Whether that is constantly learning new things, or never forgetting what you know, especially when it is hardest to practice. Here are three disciplines:
* Go Slow - Often it is better to take your time, making sure you do every step correctly, rather than rushing for a temporary spike in productivity.
* Do it Right - Never stop practicing the things you know that work. Don't stop testing first. Don't stop refactoring.
* Being Methodical - Be thorough. Be certain you are producing the best work that you are capable of through a steady and unflinching practice of virtues.

* Lists - A Craftsman knows his own productivity, and can gage how much he can get done in a certain time span. Making lists, getting metrics, and measuring your productivity will help to accomplish this.

* Understanding the Real Business Intent - It is important to remember you aren't just writing code to fulfill some requirement on a note card, but you are actually creating a product that a business intends to use. Try to understand what that note card means to them as you transform the requirement into a feature.

* Recognize your failures of the past so that you can move forward on a new level.

* Don't be shy with your ideas - Throw your ideas out there, and then be the biggest critic of them. You only stand to gain when others see your ideas (unless youzz crazy).

* Lessons always come at a cost - Stay positive, because if you are getting punished by your mistakes, remember that you also learning from your mistakes.

These are the big qualities I took from the reviews, and although I am sure there are others, this is a solid list to begin modeling my character against.

Day 42

Alas, I missed my opportunity to blog last night. At least, however, I get to blog on this day of all days. Somehow, this day is the answer to life, the universe, and everything! Ok, enough on that.

Yesterday and today I learned a lot more about how the client's system really works. There is a huge amount of depth to it, and I have now dived fair deeper into its inner workings. Right now I am charged with the task of figuring out how the system sends out mass emails, or individual emails triggered from particular actions, like signing up.

There are 3 main components to this functionality and they are:
Redis - A non-relational database that stores data using simple key-value pairs. The data can only be searched for using the keys.

Resque - A delayed job system. It has a queue which you populate with some tasks, and when it has the chance it runs through the queue performing actions of the items.

ExactTarget - An email client which can be used to send out large quantities of emails, store templates, and keep metrics on the emails it has sent.

In conjunction, these systems create a convenient way to send out emails to user's for a variety of reasons.

Basically, the way it works is the Resque system populates a queue in the Redis database with tasks for ExactTarget.

So when a user, say, signs up for a membership, they need to be sent an email confirmation. You don't need this to happen immediately, but you do want it to happen within a few seconds. You don't want the sending of the email to get in the way of maybe the user refreshing the page or something. So you call Resque.enque( and then some task) so that the task will be put into a simple Redis database that can be accessed later. Once there is some extra time, the Resque system will start up a worker which will loop through all the items in the queue can calling a perform() method on the particular class or module which represents the task. The method, in the case of the email, will then gather up the needed information, package it, and then ship it off to the ExactTarget handler. ExactTarget will then begin the process of sending out this email.

There are lots of little steps that happen in between, like transforming the data to a fro xml and json objects, but this is the general gist of it.

Over all a relative interesting process to conveniently ship out emails attached to specific actions. It seems that almost all of the complexity is in the email client, and Micah has been cooking away at that. Hopefully be will be able to implement and expand upon the existing system in just a few days.

Day 40

Wow, its Day 40 of my apprenticeship. Pretty awesome. I must say that looking back on all the things I have learned and in what ways I have grown, this apprenticeship has been one of the most, I want to say, 'constructive' experiences of my life. I am still trying to adapt to my new circumstances, after drastically changing everything in my life in a matter of weeks, but there has been huge growth stemming from my new challenges. Micah has been an excellent mentor, even though he is always busy, he still finds time to work with me on occasions to make sure I am still progressing. I have also been exposed to a myriad of different customers, businesses, business related issues, CEOs, CTOs, projects, challenges, and coding environments. Honestly, myriad isn't really the right word, but I hear it used so often these days that I must add to the disaster.

I would say if I had to name 1 thing I have learned the most about so far, it would be how much I don't know about. How much is out there, how far behind I really am, and most importantly, the things I need to start doing and learning to catch up and get ahead.

Today at work I spent a lot of time messing around with cucumber features trying to get them all to work. Honestly, I feel like I wasn't as effective as I should have, and what I learned seemed uncritical. I did, however, get to speak with the CTO for quite some time, learning all about the path their company went through, what obstacles they faced, and some important things that made them stand out. One example would be, keep it simple. There has been a huge amount of research to suggest that people really don't know what they want. In fact, the more choices people are given, the less happy they tend to end up being with their choice. Barry Schwartz and his book The Paradox of Choice: More is Less explain this quite clearly. Also Dan Gilbert explains how when you have the option between many things, the thing you choice becomes less valuable than if you were just given something with no choice. To get the context of all this I suggest you watch these TED talks: http://www.ted.com/talks/barry_schwartz_on_the_paradox_of_choice.html
http://www.ted.com/talks/dan_gilbert_asks_why_are_we_happy.html

Also, this CTO got to meet Malcolm Gladwell, thus me and Malcy are now buds one connection removed.

I was able to finish more of the rails book today, now familiarized with the concepts behind layouts, models, controllers, and views to some extent. Migrations are far more clear, and at last I understand the full and awesome power of validations. In fact, after reading a bit more in the rails book, I was able to make more project on my story for the client. I got write some sweet tests using some lambdas as well as grasp the purpose and intent of a far greater portion of the code. It seems like every 10 pages I read I gain 10% more understanding! If my math is bad enough, by the time I am done, I will have 100% understanding of rails!

Day 39

A frustrating day, much of which was spent overcoming small obstacles which took far longer than they should have, like trying to merge 2 different basecamp accounts, or getting intellij to work with cucumber. After working through all of it, I was able to spend a lot of time exploring the clients code. Still trying to get a feel of their project along with rails. I find it frequently difficult to tell what is a part of rails, and what is a part of their code. This is why it is fortunate I finished another 30 pages in the Rails book today.

To be honest, I had never really read a language specific programming book through and though before, I had only read books on general programming theory and philosophy. I had also used them for reference and such, but I was really missing out. Its pretty easy to see how much can be learned by not just reading the book, but working through the examples along side the reading. It is also very engaging! Typically I had been sleeping on the train to Chicago, but this morning I was just captured by the book and couldn't stop soaking up what it had to offer. I hope tomorrow I am able to get at least another 30 pages done, fitting in time around all the stuff I have to read for school =/

Much of the reading today was about the Models and working with databases. I found myself experimenting with every route I could find rather than just following the book's guided path. Though, this slowed me down, it also gave me a greater feel for the databases. Trying out both MySQL and SQLite, as well as looking back on a project where I used CouchDb.

CouchDb is pretty awesome btw, but I most certainly have to do more research of my own on it. I know its powers, but I don't fully understand its foundation in detail.
Anyhooo, it is time for sleep so I can work on the train again tomorrow!

Day 38

U.S. Hockey team lost to the Canadians... I know it's their major sport and all, but I still wanted to beat them again, especially when it counted.

On another note, this weekend I got some time to first, code with Micah and do some re-factoring. He, like my dad, does this very methodically and linearly. Its interesting because it always seems like they know exactly how any re-factoring is going to end up, no matter the complexity. This is not the case however. I am quite certain that, although sometimes they might have a feel for what it might end up looking like, or at least catch a whiff of the code smells, most of the time they just start cranking away and out pops a far cleaner chunk of code. The reason it always seems like they know what it will look like in the end is that they always know the right steps to take in order to get there.

On saturday Micah and I were looking at this beastly method I had written. I had briefly tried to re-factor it before, but when I attempted to extract out some smaller methods, they required about 6 parameters each! After reading Clean Code I know that methods are better off with fewer parameters, and if you are squeezing in more than 3, something is wrong. I decided that maybe I should be making a class or method object out of this function, but i hadn't yet gotten around to it.

When Micah looked at it, he didn't think it merited a new class, and instead tried to find good chunks to extract out. After a few moments, he decided to extract out the same chunks that I had tried. The difference was that he wasn't at all phased by all the parameters these methods would need. He motored on. He began finding simple ways to pull in the parameterized data, step by step reducing the number of inputs to the method. Once again I was a bit concerned because as we pulled more code into this method, to reduce the param count, I began noticing we were creating some duplication. But Micah motored on.

Finally, once we had reduced the input params down to 3, we looked back at the original method and extracted out another chunk of code. The same thing occurred and there were just too many inputs, except that this time I could see that not only would we be moving a lot of the parameterized data into this new method, we would also be pulling it out of the old one. Slowly but surely there was less and less duplication, and the data was being moved to where it actually belonged. Rather than calculating a huge batch of variables at the start of the method and then passing them into the algorithms, we would be calculating only the needed variables at the needed time in their most suitable location.

In the end the method was drastically simplified, something that always happens when pairing with someone like Micah, and we could finally see what was really going on.

What was important about this process was learning to persevere. I think one of the differences between and experienced coder and a novice coder is that even though neither can always see the end result, the experienced coder doesn't get worried when something gets uglier before it gets cleaner. They keep the goal in mind, and they take the familiar steps that usually lead to a successful result.


This weekend I also had some time to progress further with the Rails book. One basic step I thought was important is what to do when starting a project. It seems to me that you have 3 options when starting a new project. You can either jump right into it and code without any planning, you can try to plan it from start to finish with a detailed process, or you can just sketch out some rough ideas about how you think it might all work. Experts in the field seem to suggest that you should just make a rough sketch of what you are doing since one, it is useful to have a picture for both you and the customer, and two, because projects are always going to change from what you originally imagined.

The Rails book suggests that you make quick sketches with paper and a pencil to layout the feel of the project in a very broad sense. This will help you see eye to eye with your customer, and it is easy to throw away.
Time to get back to coding another rails example project!

Day 37

I apologize for being late, last night I got caught up in my Sociology hw and I completely forgot to blog. Yesterday we committed out work for the completed feature, and it got deployed in the real site. Pretty cool how fast we were able to add a new feature, and then see it already in place on their website. We actually got the feature done so fast that they just weren't ready for it, which I believe is what Micah was aiming for. Suddenly this thing that they thought would take a couple of weeks to finish was done in three days, and they were completely surprised.

This of course was the good sort of surprise, and they quite quickly brought in one of the people who would be using the feature often so that they could see how it would work. Unfortunately I had to go to class right as Micah was demoing it, but it went well and apparently they were so impressed that they included it in their deployment right away!

The client does continuous deployment, and each week they push up to the main server, so every time anyone commits code they have to make sure everything still works. It seems kind of scary, since you could potentially deploy a secretly bugged build up to the live site, but of course you can always roll back if that turns out to be the case. It does, however, let them very rapidly develop their site and there is always something new for people use and take advantage of.

Some interesting things I had never though of occurred when we were setting up the error messages that the user might see. First off, since typically the business logic belongs in the models, we wanted to be passing the error message from the model through the controller. The client had, however, been using generic error messages located in the controller. If some request failed, they would check to see if the model had populated an object with any error messages, and if there was one, they would print out a generic. We wanted to print out a more informative and meaningful error, but here is the rub - if you are making an international app, you want to have as few error messages and such as you can so that translating all of them is a straight forward process.

Day 36

Diving deeper into Rails. There is a framework used by our client which is both fairly impressive, and quite complicated. It is powerful in that it handles mostly all of the needed CRUD for controllers, with a very polymorphic and expansive system, but it is also enormously complicated an difficult to understand. Diving into the code certainly helps, but it sacrifices clarity for density. None the less, seeing the bare bones for a new framework which, once understood, can drastically increase your productivity, is pretty cool. I wish I had been there to see its development from birth.

Working more with Micah today, we got a lot done on our current feature. I revamped my Cucumber knowledge, getting to write a few step definitions and acceptance tests recently. Cucumber reads quite naturally and is pretty easy to work with.

I also learned more about the REST convention, and following it conventions for URLs and routing. This, along with defining a separation between all the dynamic structure of a page (like the Java Script) and the actual html for the page. The client, in the spirit of being Restful (btw, Rails 3.0 is going to incorporate this by default), likes to keep a web page as pure html and thus have all the JS in header operations before the page's html is constructed. In this way, if a browser doesn't accept JS, it can still read the page, its just that requests that require JS wont do anything.

Got another 30 pages done with the Rails book as well. At this point, mostly everything learned is learned through example and practice, thus I am writing a lot of code while reading. Naturally, I am making the pet project that the book instructs you to make, but along side that I am going to work on my personal website so that I can really put what I learn into practice. It is very important that I design a Rails App from start to finish on my own so that I can develop a better understanding of the separation between the client's code and built in Rails features.

I find that it is far more important to understand what is going on, than to merely know what is going on. The first is far more challenging, but also much more powerful. It is just like highschool and college physics. You can memorize all the formulas and equations you want, but come test time if you don't understand the material, you can still get caught up by a tough word problem. Understanding gives you even more than this, since if you understand the fundamentals of a formula or a problem, then you can recreate the needed equations on the spot without having them memorized. The same is true with coding, algorithms, and syntax. You can memorize and know all you want, but until you understand what you are working with, you wont be able to design creative solutions.

Sorry for being brief, I have some Sociology hw to do, and I NEED sleep! Early morning trips to Chicago are taking their toll

Day 35

Another excellent day in Chicago. Today I got to meet the CEO and founder of the current client. Our introduction was rather brief and, as one would expect, we got straight to business talking about the details of the first feature we are making for them.

One of the more interesting issues tackled today was a race condition where several users are trying to access the same expiring option at the same time. Say we have this sweepstakes going on, and only 10 people get to sign up for. This sweepstakes is released on twitter, and immediately 50 people apply to sign up, all at virtually the same time. You have a database which holds only ten redeemable account numbers, but this database is being accessed by several servers at the same time! On the very last redeemable account, 5 people request the server at such a close interval that each of their servers request the database for the same account at the same time. You can say that the last account should go to the first request for it, but what these requests are being done in parallel, so the first request in might be the last request out. Who gets it? How do you check too see if there is still an account remaining while other servers are grabbing the last account?

Turns out that Ruby with MySQL has a very nice lock! method that can be called in a model to guarantee only one write can be made to a certain row in the database at any given time. It does something else, but let me query you first. Is that enough?

Even if only one of these 5 servers was able to write to this database row, thus editing it to grabbing the last account, the 4 other servers could still read the row and therefore determine (if reading just before the row was being edited, but after it was locked) there is still an account left. The remaining servers might then try to edit the row the moment the lock was released, which could cause some issues.

Its not enough to simply lock the writing of this row, but you also must lock it so that nothing else can be reading it while it is being written to. In this way, the first of the 5 servers would lock the row, at which point the other 4 servers would have to wait until the 5th server officially had the last account before they could read the row.

Fortunately this is what the lock! method does, when using the LOCK IN SHARE MODE. This along with using a Ruby transaction (which is an all or nothing access for an object), quite nicely resolved this race condition.

Micah wrote up a script to test this (because unfortunately rspec performs its tests using transactions rather than actually accessing the database, thus making testing this race condition virtually impossible in the typical form) which would create 5 Ruby forks (not threads since Ruby threads all run through the same interpreter, and thus are really actually just one thread [ Not in JRuby though!!]) which would all request the last redeemable account at the same time. We confirmed that it worked by seeing a couple of forks grabbing the last account while only using Ruby transactions, and then confirmed the lock! worked by running this script again and again. Micah actually did a cute little command to run the script 30 times, and it never failed even through that. Though it seems nearly impossible to be certain that it worked, it seems quite likely that it works.

Start reading the Agile Web Development With Rails - Third Edition, and got about 30 pages in tonight. I am really glad I started reading this after creating my Socket Server because not only does it solidify my server knowledge, it also gives me a new realm of connections to make so that I better understand how Rails really works.

A quick overview of what I have gone through so far.
Chapter 1 was mostly all intros and talk about how Rails have developed and how this book will be structured, so never mind that. Chapter 2 is where the details begin.

Chapter 2 revolves around explaining MVC.

MVC is of course the Model View Controller framework that Rails, along with many many other applications, uses.

2.1

MVC - Model View Controller

Model - handles all data, data interaction, and business rules. Gate keeper and data store.

View - UI, normally based on model. Never handles incoming data. Different views for same model.

Controllers - The corner stone of application. Receive requests, handles them using the models, display results using views.


Handling requests:

http://my.url/store/add_to_cart/123

store - name of controller
add_to_cart - name of action
123 - extracted into id

The controller find's the user shopping cart and product (model)
Tells the cart to add the product
Shows updated info to user

2.2

Models--
Organizing data (ObjectOriented vs relation databases):

Database-centric - Bringing data and database modifications directly into production code. Ewwwwww

Object-Relational Mapping - Uses OO to control data. Maps Classes to tables, objects to rows, and columns to attributes.

Active Record - Follows the ORM model.

2.3

Views --
Often just batches of HTML, but typically wish to have some actions. Actions provided by action methods in the controllers.
Dynamic content done in 3 ways using templates:
ERb (embedded ruby)
XML Builder
RJS (JavaScript) - Ajax

Controllers --
Responsibilities - Routing external requests (friendly URLs), Managing Caching, Managing helper modules (for views), Managing sessions


Certainly a quick summary of the chapter, but if you want to know more details, the book itself it a pretty smooth read while retaining high content.

Day 34

Today we were at their office in Chicago meeting all of the guys of the new Client, getting the source code up to speed on our computers, and learning what we will be doing for them. I must say, a pretty amazing company. Their relaxed, but hard working on target mindset is extremely refreshing and conducive to an effective working environment. Their website has been wildly successful and is saving tons of people tons of money, while bringing companies tons of customers and tons of money. Where's the fault? It actually comes in when certain companies start saying "You need to set a limit! Too many customers!" Ayyy, there's the rub.

Getting started on a new existing Rail's project is always a pain. Even more so than Java or plain Ruby projects. On every Rail's project there is a new set of 10-20 gems and plugins that have to be settled. Each gem with their own specific version, frequently not the most up to date version either. On top of this, there is a wide selection of databases that can be used, and always require some love and finagling to make them happy.

I spent around 4-5 hours today getting my situation up to date and on track with their environment. Micah was able to get going in about 2 1/2 or so hours, so he got a good look at the code and a fair feel for what we will have to do to complete the feature. On a happier note though, the guys here said we got up and ready faster than anyone else before us, who often took up to a week!

Most of my time was spent getting MySQL working. I jump from error to error, then fixed stuff, but went back to the same error as before. It mostly all stemmed back to not being on Snow Leopard, but the default download for mysql was using the x86_64 archive. Later on I figured this out, went back and got the right version - i386 archive, but still had issues with missing refs.

One really useful tool I learned about is the RVM. Ruby Version Manager. It can switch back and forth between different Ruby environments, with different versions of Ruby (like the core Ruby or Ruby Enterprise Edition (which is mega faster)), different gems and gem versions. With just a simple command like: rvm ree I could switch to Ruby Enterprise Edition with all the gems I needed for this client, and then with: rvm system I could swap back to my normal Ruby environment. Uber powerful.

Tomorrow we will start planning out our iterations and our story cards.

Day 33

I finished the HTTP server on time for my demo on Friday, which went smoothly. This week Micah and I will be in Chicago doing some work for a web based company. Their website hosts coupons for businesses. A business, say Gino's East will post a possible coupon on their website for maybe 50% off a large pizza, and if 300 people sign up for the coupon, then Gino's will carry through with it. Anyone who wants the coupon can sign up for it on the day it is hosted, but tomorrow brings a new coupon and the end to the sales of the old one. They have been very successful, and their site is pretty awesome.

Ok, I will just share some of the basic tid-bits of how the server worked.

I made a very very basic site for the server to host in order to test to make sure it worked. This site had certain requirements which my server had to support. It had to have a few basic pages (like the initial page '/index.html and a homepage), catch incorrect URL requests and respond with a 404, support common MIME types (images, pdf, even small video files), have an Echo page which would spit back out the arguments past to it via the URL, and handle several requests at the same time.

Hosting the basic pages was fairly straight forward once you know how to format a request and response. The most basic request header looked like this:

GET /index.html HTTP/1.1
Host: localhost:80

The form is Request type page requested protocol
Host: resource name : port

The resource name could be localhost or could ben something like www.google.com.

The most basic response header would look like this:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 500

The form is Protocol response code and response type (for an error page the code would be 404 and type Not Found)

Typically there is a flurry of other information like cookie and browser information. After the header would follow the body of the response.

For my home page, and like I said... super basic pages, the response body would just be some html like
<%html><%h1>This is the home page!

Sweet website right?

But then for a page with an image or pdf there were a lot of extra steps. First, in the response header you have to figure out what the page's MIME type will be, and then you have to get the size of the page. An image would be like image/jpeg or image/gif, and then you send the size viz the Content-Length param.

Once you ship the header, you have to send the response body, which will contain the image file. TCP/IP sends all files in little chunks, so you can't just ship the whole image at once. You have to break it up into individual bytes and stream it onto an output stream. This was a bit of a pain at first because even with some of the built in Java support for these IO streams, it can be a bit tricky figuring out which types to use, and how to make all the needed conversions.

For example, to send an image as a byte stream given only its location there are several steps you need to take. First, in Java, you need to create a File object for the image. For this, you need a File Output Stream to read the file and put it into local memory. Then you need to turn that file into a byte stream, and get its size. Finally you write that byte stream into a Byte Array Output Stream, which can then ship the bytes onto your Socket's output stream.

A fairly lengthy process to merely ship photo's from computer to computer, but it is reliable and will work with virtually any file type.

At last, making an echo page is pretty simple. When you receive a request with some input params, they are tide into the URL something like this:
www.justinmmartin.com:80/index.html?a=1&b=2&c=3

where a,b,c are the params. The request has the params in the initial GET line:

GET /index.html?a=1&b=2&c=3 HTTP/1.1

So you can just parse out the string, snatch up the params by splitting the string on certain characters, and then add them back into HTML to be shipped out in your response.

A relatively broad summary for what needs to be done to create a basic server, but hopefully you get the picture.

Day 32

I learned a LOT today... well yesterday, last night, and this morning. Basically, the last few days I learned all about how to make a Socket Server and how HTTP requests work, but I really didn't know how to make them happen. Today, I learned that. There is a massive difference between understanding the principles behind how something should work, and the understanding required to actually make that something work.
Today I learned how an HTTP server, and all the communication involved in the process, works - in practice.
I learned how to form real, and usable requests and response.
I learned how to use them to display different types of html pages.
I learned how to link them to a file system, such that a request routes directly to a directory, and from it, derives the appropriate response.
I learned how to run multiple connections.
I re-learned how to make a build.xml file to run an ant process so that I can snatch out a .jar file.
I learned about pretty much every type of input and output stream Java has to offer, as well has how to link them up in unison to move files around in efficient manners.
I learned how to use chunked response, although not in practice... But i am pretty sure I could put it into practice in a reasonable amount of time.
I learned how to send different MIME types across the web so that I can display images and such from a local file onto a browser.

And I learned a whole lot of other stuff in order to learn how to learn what I needed to learn! This is likely the most important thing I derived from the long day of hardship.

Here is the rub... its now like 4:15 AM and I am not quite finished implementing all the mime types (though pretty close) or being able to pick up parameters (another fairly short matter). I gotta have it done by about noon tomorrow so... Imma sleep and blog more this weekend when my brain can recollect details.

P.S. telnet is pretty sweet terminal command. check it out