Tuesday, October 20, 2009

Keeping Seaside session alive

I've created a simple Seaside app to monitor the state of our servers. It does this by screen-scraping multiple service status webpages hosted on our servers. The monitoring was coded as a sUnit set of tests which can run standalone or as part of the Seaside application. Any service breaches are sent via Growl or Snarl. All status responses are persisted using Magma and so far it's been working well.

There have been a few minor issues though.

Memory consumption was a little high and I pinpointed it to Magma + Squeak. It seemed the MaTransaction's and the oid's were not being garbage collected even though they were in a WeakDictionary. I fixed this by registering the WeakDictionary with the WeakArray finalization process in MaTransaction. Memory problem fixed :)

Another issue was that sessions were expiring after 10 minutes and since we typically look at the results every 2 hours or so. Reconnecting to the Magma store and re-establishing the session was slightly slowing down the responsiveness of the app, so I decided to add a Javascript session keep-alive hack. I created a little Seaside component that I render in my main Seaside component. It's renderContentOn: method reads
>>renderContentOn: html
(html periodicalEvaluator frequency: 5 minutes asSeconds) callback: [:renderer | "do nothing"].
All this does is call back into the app every 5 minutes and now the responsiveness has improved and we still have session management (when the browser is closed).

The other issue was really to a poor design decision on my part. The status responses received from the servers are persisted using Magma and I had created a few indexes and used a MagmaCollection. One of the monitoring pages show the total number of tests and the total number of failures. This query was progressively getting more expensive as the status response objects were being materialised to see if the response was successful or not. The answer to this was to create another index for this boolean. So I created a BooleanIndex added it to the MagmaCollection and bam, much better performance. Queries are now coming back in under 1.5 seconds whereas before it was taking about 90 seconds. Magma DB has about 20000 entries at the moment, so I'm pretty pleased with the response times.

Overall I am pleased with the use of Seaside, Magma and Pharo. What I want to do now is make the site look better and maybe add some ajax calls to make the site more usable. But then again this is a hack, maybe I'll live with it.

Tuesday, September 22, 2009

Snarl Smalltalk client

After basically hacking a simple Smalltalk Snarl client I decided to put it to use.

First of all a quick status on the client:
  • it covers the bare minimum functionality,
  • uses the network protocol because it was easy to do,
  • has no error handling because most of the time it just works.
To monitor our systems (we have about 9 servers) we use simple web interfaces to query their status. Normally we would log in to each app via HTTP and use the web interface, so I decided it would be easier to use Snarl to notify us as soon as something goes wrong (we have monitoring software but there are some issues ... anyway). I created a simple Seaside app to register/de-register a Snarl service running on our client PC's, and also start a background process to interrogate the HTTP interfaces of the PC's we were monitoring.

So far it has been working really well without any issues. As this was an interim measure I didn't want to spend too much time creating a monitoring tool but so far am impressed with how simple it was to hook everything up.

Some technical details on the solution: use SUnit to test the REST-ful services, announce the results (using Announcements) so that either the TestCase runs standalone if there isn't an announcer registered or it notifies all subscribers. Set up UpsideDownSnarl client to process the announcements. Used Seaside to create a management interface.

The sweetest section of code in all of this: Using Process>>allSubInstances and then using the enumeration protocol to find and terminate any processes that were registered on the clients behalf.

Smalltalk makes it so easy to do something like this :) you gotta love the quick feedback cycles

UpsideDownSnarl - Smalltalk Snarl client

I need to monitor some services at work and they've been misbehaving for a while now. So I saw James Robertson blogging on Snarl/Growl and decided to hack my own Snarl client for Pharo/Squeak.

I uploaded the first version to SqueakSource and I must say Snarl is a nifty little tool. I now can run my Snarl client + service tests and get notified if something goes down.

Sunday, March 15, 2009

Qi4j == Magritte + Traits + OODB + Newspeak

I recently came across the opensource Qi4j project which has Rickard Öberg co-founder of JBoss as one of its core developers. Now I haven't gone into a large amount of detail, but I have read the tutorial and some examples and although I see the benefit, I feel Java as the implementation language was definitely the wrong choice.

If you are going to try predict the future and the best way is to invent it then why cripple it with Java? Ok so thats a little harsh but look at the examples and tell me if you can see the domain code easily, Java is just not expressive/malleable enough to make it read well and Qi4j's meta-descriptions (as annotations) are not extensible or dynamic. In my opinion this limits its power.

Qi4j == Magritte + Traits + OODB + Newspeak

Magritte
...is a meta-description framework, describing domain classes and their respective attributes, relationships and constraints.
With Magritte you can describe your domain model similar to what Qi4j is doing. A major difference is that Magritte is self-describing and the meta-model is modifiable at runtime. So since you can describe your domain you can do things like object persistency, indexing, reports and views etc. and having this meta-model would allow one to create an optimised OODB (ROE?, Gemstone) persistence facility or a relational database mapping scheme (GLORP).

The mixin and behaviour separation constructs in Qi4j remind me of traits which arguably is a better form of behaviour composition.

The Qi4j application 'assembly' steps remind me of Newspeak's nested classes and module system.

I appreciate the ideas that Qi4j is trying to achieve but believe they are most likely wasting lots of developer 'cycles' trying to wrangle Java rather than coming up with some innovative ideas. A dynamic, fully reflective OO lanaguage (like Smalltalk :)) would be better suited to their needs as it would allow them to focus on their 'domain' which is building a DDD enabling framework.

Friday, March 13, 2009

Are you an OO Bigot? Time for a different modeling technique maybe...

After reading through a blog entry which basically centered on 'Reducing cost by using OO principles'
it made me realise that there is the danger where we stick to our ways and preclude lots of other valid design and modeling choices. The author describes how he has been described as 'over object orienting' solutions. I feel it happens to many of us, and when people are telling us this it may be a valid flag that we are over-generalising and therefore over-complicating our design
i.e. typically the more generalised the solution the more 'mental hoops' the readers of the code will have to go through to understand it. It might be time to use another modeling technique, function oriented, rule based etc.

BTW I'm not implying that the referenced blog entry is wrong in saying that OO design can reduce cost but what I do think is that cost can be measured in many ways.
Yes there are cost 'reductions' in being able to localise code change as to me one of OO designs strength's lies in the ability to package behaviour and state.

This 'hopefully' reduces the cost of change as the state changing behaviour should be localised in a few key areas (depending on how deep the change 'cuts across the grain').

But here lies the rub; OO works well when the object model lies close to the actual real world 'object' or metaphor that it is modeling.

What I mean is that OO designs favour mental models where we can safely imply what the expected behaviour is of an object and not have to dive into the details.
In this way users of our code can safely modify the code (tests help to verify that the known behaviours still work after the change but they don't verify all possibilities, just the ones the tester deemed important).

OO design does not lend itself to composing of behaviours because it has to typically deal with state. Users of the object need to understand the messages that the object can react to and what the intended result of the message is. They don't need to know how it is done just that it will be done, but this limits how you can compose behaviour because the timing of calls is important.

Functional languages and designs would (and by no means am I an expert) favour applying functions to a data structure (and to other functions ala higher order functions) that transform the data safely and typically are side effect free.
This is crucial to being able to compose behaviour as you can apply the function to data over and over again knowing the result will be the same. Nothing changes that behaviour, there is no concern of state to add to the stack of your 'conceptual model'.

I believe (and not from first hand experience) that it falls down somewhat because it lacks powerful logic structuring mechanisms. i.e. it's hard to keep track of the transforms and group/package them, the function may be too coarse and have to be broken down into 2 or more functions; or it may be too fine grained and therefore be more difficult to understand how the transforms fit together i.e. to get a desired result which set of transformations should I use.

Ok example time, I'll use working with collections of objects in Smalltalk and Java
Say we have a business requirement to contact patients who are in arrears at a medical practice.


kindReminder := [:account | account dueDate > (Date today + 30 days)].
slightlyOverdue := [:account | account dueDate > (Date today + 60 days)].
veryOverdue := [:account | account dueDate > (Date today + 90 days)].

'this is a function that emails a contact with a message'
email := [:contact :message | Email mail: contact with: message].
'email the patient'
emailPatient := [:account | email value: account contact value: 'overdue'].
'leave a voice mail on their phone'
leaveVoiceMail := [:account | PhoneService leaveVoiceMailFor: account contact saying: 'overdue please call us'].
'apply the normal email function to the patient and then make the lawyers aware by sending the same email sent to the patient'
emailLawyers := emailPatient ensure: [email value: lawyer contact value: emailPatient value asMessage].

'apply the functions to the data'
Accounts findAll select: kindReminder thenDo: emailPatient.
Accounts findAll select: slightlyOverdue thenDo: leaveVoiceMail.
Accounts findAll select: veryOverdue thenDo: emailLawyers


OO would be cleaner to the eye, we could do something like:

List inArrears=Accounts.findAllInArrearsByAtLeast(Days.from(30)); // 30 days
for (Account account : inArrears) {
account.notifyInArrears();
}

public void notifyInArrears() {
ContactStrategy.createArrearsStrategyFor(this).contact();
}

// ContactStrategy factory method would create the relevant strategy based on the account dueDate etc


The 'functional style' (kind of, it still is Smalltalk) is more amenable to composition of behaviour safely by using higher order functions.
The OO style is separated cleaner so change is easier to localise but extension points are harder to 'mould' in unless hooks were provided already.

Now to get back to some kind of takeaway from this rambling post :).

OO is great to factor behaviour;
...In Smalltalk, nothing ever happens here...
'nothing ever happens here behaviour' (I think Adele Goldberg said that about Smalltalk/OO design) though means that typically understanding the model is more complex. There are just more moving parts with state and everything seems to happen elsewhere.

Functional design favours transformations where we know what the input and output are meant to be, so we can compose behaviour more easily which is a very powerful tool.

I feel a blend of styles would be best, something akin to the Smalltalk example but putting most of the logic behind intention revealing methods, and using the powers of functions (closures) or even Traits — Composable Units of Behavior to compose certain behaviours.

The ultimate goal for me when designing a solution is that it must be readable, understandable and therefore easy to change.

Sorry about the long rambling post...if you made it here...well...wow

Tuesday, March 3, 2009

Smalltalk: power of simplicity and personal mastery

One of the principles stated in Design Principles Behind Smalltalk by Daniel H. H. Ingalls which appeared in the BYTE Magazine, August 1981 was that the Smalltalk project had a bias towards:
Personal Mastery: If a system is to serve the creative spirit, it must be entirely comprehensible to a single individual.

Well my Smalltalk is not great, I have read some of the free books on Stephane Ducasse's website, and yet I still find Smalltalk to be intuitive and easy to do things with. I'm not sure if it was the way the language was designed, the class design (libraries) or a combination of the 2.

Let me give a simple example, but first a disclaimer; I could have used excel or some other mechanism to do this but it was more of a learning experience.
Ok so basically I had 2 CSV reports that i had run on 2 environments that I wanted to compare to make sure that there were no differences. Thats really simple and I didn't explore many options, I just fired up Pharo and started typing into a workspace:

firstReportList:= (CSVParser parse: (FileDirectory default readOnlyFileNamed: 'resources\first.csv')).
firstReportList := firstReportList asOrderedCollection.
firstReportList allButFirstDo: [:entry | entry at: 5 put: nil.].

secondReportList := (CSVParser parse: (FileDirectory default readOnlyFileNamed: 'resources\second.csv')).
secondReportList :=secondReportList asOrderedCollection.
secondReportList allButFirstDo: [:entry | entry at: 5 put: nil.].

Transcript clear.
Transcript nextPutAll: (liveList = wareList) asString; cr; flush.
Transcript nextPutAll: firstReportList size asString; cr; flush.
Transcript nextPutAll: secondReportList size asString; cr; flush.


CSVParser was a small nifty library by Avi Bryant that I had previously loaded in my image. The line firstReportList allButFirstDo: [:entry | entry at: 5 put: nil.]. was put in to ignore the heading on the first line and then for all the other rows blank out (by making it nil) a column that was deliberately different in the 2 files with which I did not want to compare.

Now firstly how sweet is that!
Secondly I didn't get any differences so I was happy with the code i.e. I didn't have to write code to spit out the differences although I feel that would have been trivial.
Thirdly try doing that in my staple language Java...umm no thanks...I know its possible but not as succintly as Smalltalk. I'm sure Lisp or Perl would have a nifty way of doing this, and probably Smalltalks distant cousins offspring Ruby.

I can't wait for what Daniel Ingalls ended the article with (I think many people want to be part of this too :) and its not Ruby )
Natural Selection: Languages and systems that are of sound design will persist, to be supplanted only by better ones.


On another language note I see the Lift web framework written in Scala was released, I took a very brief look at some of their examples and I was not impressed with the mental hoops it took to make it do something simple. Seaside seems simpler to grasp and as expressive if not more....but I'd still need to look at it in more detail.