point of view is worth 80 IQ points...

Keeping Seaside session alive

2009-10-20T21:13:00.002+02:00

I've created a simple Seaside app to monitor the state of our servers. It does this by screen-scraping multiple service status webpages hosted on our servers. The monitoring was coded as a sUnit set of tests which can run standalone or as part of the Seaside application. Any service breaches are sent via Growl or Snarl. All status responses are persisted using Magma and so far it's been working well.

There have been a few minor issues though.

Memory consumption was a little high and I pinpointed it to Magma + Squeak. It seemed the MaTransaction's and the oid's were not being garbage collected even though they were in a WeakDictionary. I fixed this by registering the WeakDictionary with the WeakArray finalization process in MaTransaction. Memory problem fixed :)

Another issue was that sessions were expiring after 10 minutes and since we typically look at the results every 2 hours or so. Reconnecting to the Magma store and re-establishing the session was slightly slowing down the responsiveness of the app, so I decided to add a Javascript session keep-alive hack. I created a little Seaside component that I render in my main Seaside component. It's renderContentOn: method reads

>>renderContentOn: html
(html periodicalEvaluator frequency: 5 minutes asSeconds) callback: [:renderer | "do nothing"].

All this does is call back into the app every 5 minutes and now the responsiveness has improved and we still have session management (when the browser is closed).

The other issue was really to a poor design decision on my part. The status responses received from the servers are persisted using Magma and I had created a few indexes and used a MagmaCollection. One of the monitoring pages show the total number of tests and the total number of failures. This query was progressively getting more expensive as the status response objects were being materialised to see if the response was successful or not. The answer to this was to create another index for this boolean. So I created a BooleanIndex added it to the MagmaCollection and bam, much better performance. Queries are now coming back in under 1.5 seconds whereas before it was taking about 90 seconds. Magma DB has about 20000 entries at the moment, so I'm pretty pleased with the response times.

Overall I am pleased with the use of Seaside, Magma and Pharo. What I want to do now is make the site look better and maybe add some ajax calls to make the site more usable. But then again this is a hack, maybe I'll live with it.

Snarl Smalltalk client

2009-09-22T19:45:00.005+02:00

After basically hacking a simple Smalltalk Snarl client I decided to put it to use.

First of all a quick status on the client:

it covers the bare minimum functionality,
uses the network protocol because it was easy to do,
has no error handling because most of the time it just works.

To monitor our systems (we have about 9 servers) we use simple web interfaces to query their status. Normally we would log in to each app via HTTP and use the web interface, so I decided it would be easier to use Snarl to notify us as soon as something goes wrong (we have monitoring software but there are some issues ... anyway). I created a simple Seaside app to register/de-register a Snarl service running on our client PC's, and also start a background process to interrogate the HTTP interfaces of the PC's we were monitoring.

So far it has been working really well without any issues. As this was an interim measure I didn't want to spend too much time creating a monitoring tool but so far am impressed with how simple it was to hook everything up.

Some technical details on the solution: use SUnit to test the REST-ful services, announce the results (using Announcements) so that either the TestCase runs standalone if there isn't an announcer registered or it notifies all subscribers. Set up UpsideDownSnarl client to process the announcements. Used Seaside to create a management interface.

The sweetest section of code in all of this: Using Process>>allSubInstances and then using the enumeration protocol to find and terminate any processes that were registered on the clients behalf.

Smalltalk makes it so easy to do something like this :) you gotta love the quick feedback cycles

UpsideDownSnarl - Smalltalk Snarl client

2009-09-22T00:33:00.004+02:00

I need to monitor some services at work and they've been misbehaving for a while now. So I saw James Robertson blogging on Snarl/Growl and decided to hack my own Snarl client for Pharo/Squeak.

I uploaded the first version to SqueakSource and I must say Snarl is a nifty little tool. I now can run my Snarl client + service tests and get notified if something goes down.

Qi4j == Magritte + Traits + OODB + Newspeak

2009-03-15T22:50:00.006+02:00

I recently came across the opensource Qi4j project which has Rickard Öberg co-founder of JBoss as one of its core developers. Now I haven't gone into a large amount of detail, but I have read the tutorial and some examples and although I see the benefit, I feel Java as the implementation language was definitely the wrong choice.

If you are going to try predict the future and the best way is to invent it then why cripple it with Java? Ok so thats a little harsh but look at the examples and tell me if you can see the domain code easily, Java is just not expressive/malleable enough to make it read well and Qi4j's meta-descriptions (as annotations) are not extensible or dynamic. In my opinion this limits its power.

Qi4j == Magritte + Traits + OODB + Newspeak

Magritte

...is a meta-description framework, describing domain classes and their respective attributes, relationships and constraints.

With Magritte you can describe your domain model similar to what Qi4j is doing. A major difference is that Magritte is self-describing and the meta-model is modifiable at runtime. So since you can describe your domain you can do things like object persistency, indexing, reports and views etc. and having this meta-model would allow one to create an optimised OODB (ROE?, Gemstone) persistence facility or a relational database mapping scheme (GLORP).

The mixin and behaviour separation constructs in Qi4j remind me of traits which arguably is a better form of behaviour composition.

The Qi4j application 'assembly' steps remind me of Newspeak's nested classes and module system.

I appreciate the ideas that Qi4j is trying to achieve but believe they are most likely wasting lots of developer 'cycles' trying to wrangle Java rather than coming up with some innovative ideas. A dynamic, fully reflective OO lanaguage (like Smalltalk :)) would be better suited to their needs as it would allow them to focus on their 'domain' which is building a DDD enabling framework.

Are you an OO Bigot? Time for a different modeling technique maybe...

2009-03-13T00:44:00.014+02:00

After reading through a blog entry which basically centered on 'Reducing cost by using OO principles'
it made me realise that there is the danger where we stick to our ways and preclude lots of other valid design and modeling choices. The author describes how he has been described as 'over object orienting' solutions. I feel it happens to many of us, and when people are telling us this it may be a valid flag that we are over-generalising and therefore over-complicating our design
i.e. typically the more generalised the solution the more 'mental hoops' the readers of the code will have to go through to understand it. It might be time to use another modeling technique, function oriented, rule based etc.

BTW I'm not implying that the referenced blog entry is wrong in saying that OO design can reduce cost but what I do think is that cost can be measured in many ways.
Yes there are cost 'reductions' in being able to localise code change as to me one of OO designs strength's lies in the ability to package behaviour and state.

This 'hopefully' reduces the cost of change as the state changing behaviour should be localised in a few key areas (depending on how deep the change 'cuts across the grain').

But here lies the rub; OO works well when the object model lies close to the actual real world 'object' or metaphor that it is modeling.

What I mean is that OO designs favour mental models where we can safely imply what the expected behaviour is of an object and not have to dive into the details.
In this way users of our code can safely modify the code (tests help to verify that the known behaviours still work after the change but they don't verify all possibilities, just the ones the tester deemed important).

OO design does not lend itself to composing of behaviours because it has to typically deal with state. Users of the object need to understand the messages that the object can react to and what the intended result of the message is. They don't need to know how it is done just that it will be done, but this limits how you can compose behaviour because the timing of calls is important.

Functional languages and designs would (and by no means am I an expert) favour applying functions to a data structure (and to other functions ala higher order functions) that transform the data safely and typically are side effect free.
This is crucial to being able to compose behaviour as you can apply the function to data over and over again knowing the result will be the same. Nothing changes that behaviour, there is no concern of state to add to the stack of your 'conceptual model'.

I believe (and not from first hand experience) that it falls down somewhat because it lacks powerful logic structuring mechanisms. i.e. it's hard to keep track of the transforms and group/package them, the function may be too coarse and have to be broken down into 2 or more functions; or it may be too fine grained and therefore be more difficult to understand how the transforms fit together i.e. to get a desired result which set of transformations should I use.

Ok example time, I'll use working with collections of objects in Smalltalk and Java
Say we have a business requirement to contact patients who are in arrears at a medical practice.


kindReminder := [:account | account dueDate > (Date today + 30 days)].
slightlyOverdue := [:account | account dueDate > (Date today + 60 days)].
veryOverdue := [:account | account dueDate > (Date today + 90 days)].

'this is a function that emails a contact with a message'
email := [:contact :message | Email mail: contact with: message].
'email the patient'
emailPatient := [:account | email value: account contact value: 'overdue'].
'leave a voice mail on their phone'
leaveVoiceMail := [:account | PhoneService leaveVoiceMailFor: account contact saying: 'overdue please call us'].
'apply the normal email function to the patient and then make the lawyers aware by sending the same email sent to the patient'
emailLawyers := emailPatient ensure: [email value: lawyer contact value: emailPatient value asMessage].

'apply the functions to the data'
Accounts findAll select: kindReminder thenDo: emailPatient.
Accounts findAll select: slightlyOverdue thenDo: leaveVoiceMail.
Accounts findAll select: veryOverdue thenDo: emailLawyers

OO would be cleaner to the eye, we could do something like:


List inArrears=Accounts.findAllInArrearsByAtLeast(Days.from(30));  // 30 days
for (Account account : inArrears) {
    account.notifyInArrears();
}

public void notifyInArrears() {
    ContactStrategy.createArrearsStrategyFor(this).contact();
}

// ContactStrategy factory method would create the relevant strategy based on the account dueDate etc

The 'functional style' (kind of, it still is Smalltalk) is more amenable to composition of behaviour safely by using higher order functions.
The OO style is separated cleaner so change is easier to localise but extension points are harder to 'mould' in unless hooks were provided already.

Now to get back to some kind of takeaway from this rambling post :).

OO is great to factor behaviour;

...In Smalltalk, nothing ever happens here...

'nothing ever happens here behaviour' (I think Adele Goldberg said that about Smalltalk/OO design) though means that typically understanding the model is more complex. There are just more moving parts with state and everything seems to happen elsewhere.

Functional design favours transformations where we know what the input and output are meant to be, so we can compose behaviour more easily which is a very powerful tool.

I feel a blend of styles would be best, something akin to the Smalltalk example but putting most of the logic behind intention revealing methods, and using the powers of functions (closures) or even Traits — Composable Units of Behavior to compose certain behaviours.

The ultimate goal for me when designing a solution is that it must be readable, understandable and therefore easy to change.

Sorry about the long rambling post...if you made it here...well...wow

Smalltalk: power of simplicity and personal mastery

2009-03-03T10:54:00.006+02:00

One of the principles stated in Design Principles Behind Smalltalk by Daniel H. H. Ingalls which appeared in the BYTE Magazine, August 1981 was that the Smalltalk project had a bias towards:
Personal Mastery: If a system is to serve the creative spirit, it must be entirely comprehensible to a single individual.

Well my Smalltalk is not great, I have read some of the free books on Stephane Ducasse's website, and yet I still find Smalltalk to be intuitive and easy to do things with. I'm not sure if it was the way the language was designed, the class design (libraries) or a combination of the 2.

Let me give a simple example, but first a disclaimer; I could have used excel or some other mechanism to do this but it was more of a learning experience.
Ok so basically I had 2 CSV reports that i had run on 2 environments that I wanted to compare to make sure that there were no differences. Thats really simple and I didn't explore many options, I just fired up Pharo and started typing into a workspace:


firstReportList:= (CSVParser parse: (FileDirectory default readOnlyFileNamed: 'resources\first.csv')).
firstReportList := firstReportList asOrderedCollection.
firstReportList allButFirstDo: [:entry | entry at: 5 put: nil.].

secondReportList := (CSVParser parse: (FileDirectory default readOnlyFileNamed: 'resources\second.csv')).
secondReportList :=secondReportList asOrderedCollection.
secondReportList allButFirstDo: [:entry | entry at: 5 put: nil.].

Transcript clear.
Transcript nextPutAll: (liveList = wareList) asString; cr; flush.
Transcript nextPutAll: firstReportList size asString; cr; flush.
Transcript nextPutAll: secondReportList size asString; cr; flush.

CSVParser was a small nifty library by Avi Bryant that I had previously loaded in my image. The line firstReportList allButFirstDo: [:entry | entry at: 5 put: nil.]. was put in to ignore the heading on the first line and then for all the other rows blank out (by making it nil) a column that was deliberately different in the 2 files with which I did not want to compare.

Now firstly how sweet is that!
Secondly I didn't get any differences so I was happy with the code i.e. I didn't have to write code to spit out the differences although I feel that would have been trivial.
Thirdly try doing that in my staple language Java...umm no thanks...I know its possible but not as succintly as Smalltalk. I'm sure Lisp or Perl would have a nifty way of doing this, and probably Smalltalks distant cousins offspring Ruby.

I can't wait for what Daniel Ingalls ended the article with (I think many people want to be part of this too :) and its not Ruby )

Natural Selection: Languages and systems that are of sound design will persist, to be supplanted only by better ones.

On another language note I see the Lift web framework written in Scala was released, I took a very brief look at some of their examples and I was not impressed with the mental hoops it took to make it do something simple. Seaside seems simpler to grasp and as expressive if not more....but I'd still need to look at it in more detail.

point of view is worth 80 IQ points...: JBoss HA Singleton using a service interface

2008-06-12T14:05:00.000+02:00

point of view is worth 80 IQ points...: JBoss HA Singleton using a service interface

JBoss HA Singleton using a service interface

2008-06-11T16:49:00.012+02:00

I wrote this post on a previous blog that was lost but resurrected it here:

Some related links are:
http://wiki.jboss.org/wiki/JBossHASingletonRemoteAccess
http://www.jboss.com/index.html?module=bb&op=viewtopic&t=55794
http://lists.jboss.org/pipermail/jboss-user/2006-October/014954.html

You can also access the HA Singleton via the actual services interface instead of org.jboss.jmx.adaptor.rmi.RMIAdaptorExt. So for example if we wanted to access a service with the following interface com.platform.management.logging.distributed.RemoteServerLoggerMBean, then we would use something like the following:


<server>
<mbean code="org.jboss.invocation.jrmp.server.JRMPProxyFactory"
name="jboss.jmx:type=adaptor,name=SingletonInvoker,protocol=jrmp,service=proxyFactory">
<depends optional-attribute-name="InvokerName">jboss:service=invoker,type=pooled</depends>
<depends optional-attribute-name="TargetName">
my.domain:service=HaRemoteLoggingService</depends>
<attribute name="JndiName">jmx/invoker/SingletonRMIAdaptor3< /attribute>

<attribute name="InvokeTargetMethod">true</attribute>
<attribute name="ExportedInterfaces">
  com.platform.management.logging.distributed.RemoteServerLoggerMBean
</attribute>
<attribute name="ClientInterceptors">
  <interceptors>

    <interceptor>org.jboss.proxy.ClientMethodInterceptor< /interceptor>
    <interceptor>org.jboss.invocation.InvokerInterceptor< /interceptor>
  </interceptors>
</attribute>

</mbean>
</server>

UI and domain code

2008-06-11T13:49:00.002+02:00

Although I am more of a back-end programmer I have worked with some, albeit, rudimentary webpages, recently in seaside. I've often thought of the mismatch between domain code (the model) and the view that we create around it to manipulate the domain.

What is this mismatch? It seems that there are a couple of camps on how to create an application layer around a model where either you create fine grained Transaction Script code or more coarse grained Transfer Objects with domain Commands to perform the business functionality.
The problem I feel is that business-domain-model code then moves into the application layer (most people create Commands that are nothing more than a glorified Transaction Script). The problem is your business-domain logic is in the wrong layer and therefore business rule consistency and re-use is diminished or hard to separate due to the coupling.

Trying to follow a design with a rich domain model I would say that an application should have it's own application-domain-model (idealy with its own well defined bounded context and a mapping layer to the actual business-domain-model below), and this application-domain-model would drive the business-domain-model code.
It feels like most people see MVC very explicitly and would like to have 'business objects' (objects persisted to a DB) with a view/presentation aspect and then a thin and very direct user gesture interpreter to manipulate the objects and satisfy the business rules. Now in simple CRUD applications this is fine, but when we have rich business-domain-models I can't see how this solution would scale. Abstractions would start to leak out, coupling would occur between application code and business code, and generally the 'application' logic cannot be reused/reappropriated on a different UI. What I'm trying to say is that a web app cannot be changed to a J2ME app as the app code is very specific.

Why aren't we creating an application-domain-model that abstracts what the app is trying to achieve and then we may be able to cater for different UI/interaction models over it. Also we would be encapsulating the application state/process-logic from the business-domain-model.

Some disclaimers: this is a rambling post because I'm not sure what the right answer is, all business problems are very different in complexity and so these arguments won't hold in all situations.

Funny enough what I wanted to talk about was Magritte which is dynamic meta-description framework for domain objects which helps developers to build different views, editors, and reports; and handle querying, validation and storing domain objects in an extensible and easy way.

Well I'll leave that for another post and maybe think more of the application-domain-model vs business-domain-model issue...

Immutability mutiny

2008-06-10T11:07:00.003+02:00

As part of some legacy code we designed a Date wrapper for our persistent objects, that referenced dates, so that we could hide the fact of if we were using 'varchars' to store a date or using an actual 'timestamp' in our Oracle DB. All our code would use our wrapper class and be oblivious to the underlying implementation and also provide a more 'humane' interface.

As part of this initial design back in 2002 the developers decided to make the class a value object that was immutable, exhibited side-effect free functions, closure of operations and was a stand-alone class (see DDD by Eric Evans) which made developers life easier as they could safely perform operations on this Date object without needing to know the internals or combine these objects in a safe manner. Now I am not saying that the developers designed it knowing all of this but rather that it grew from a very simple interface into something that exhibited most of these behaviours.

Now to get to the point of this post: All of us thought that our Date wrapper was immutable and we relied on this fact heavily even caching all Date's ever created for garbage collection purposes, but it turns out after 6 years of use we just found out that it in fact is not.
The problem is that our ORM (Toplink) was mapped so that it could use protected method accessors to determine the value to write and read (which supplied our polymorphic behaviour to the clients). Now if we create an object and set an instance variable to a Date and then regsiter this with a unit of work, modify the object's date, and then try persist; Toplink will check the changeset and merge the result into the original object which bypasses our immutable side-affect free functions, and directly changes the object through the protected methods.

You could say where are all the unit tests; the unit tests should catch this; after 6 years you never came across this?

But we have nearly 2500 unit tests, run regression tests using our staging code vs our live code for about 400 000 (medical aid insurance) policies we have on our systems and yet we never came across this.

The problem boils down to: for some reason all the developers followed a way of coding that never included a default constructor that set a date on an object and then registered this new object with the unit of work. What we typically did was register the object at the end as we typically had control/awareness of what was being modified created. New developers that had joined the team had not been 'tainted' with the way that things had been done and bingo things began to break horribly.

I'm not sure how we can avoid immutability issues like this when using Toplink in this manner but anyway we have an idiom now on how to use dates and some code that checks if an existing date is being modified by Toplink or a client.

Refactoring to show the rules

2008-06-09T16:35:00.004+02:00

A team mate and myself had a good time refactoring some non-OO spaghetti code into a more robust OO version. We didn't set out to do pair programming or refactor the code to the extent we did (partly because we were approaching a test freeze and did not have the time and partly because the code had been in this state for over 4 years!).

The nice end effect of this was that pair programming helped us share knowledge, not only business knowledge but also design knowledge. We constantly discussed micro-design decisions like:

I'm naming this method/variable this as it better reflects the intent...
Restructuring the code like this will have this advantage but may have these disadvantages...
If we create more classes and distribute the desired behaviour across them we will factor the code better and get rid of all these conditionals, but now this complex method has become a system of objects messaging each other. Which is more simple? ...

and so the discussions went on.

We both feel that the structure may feel more foreign to juniour programmers or to 'procedural OO programmers' for instance 1 comment was:

when debugging it is more difficult to see what is actually executing this code because of all the indirection/polymorphism

But the end product clearly factored the monolithic conditional-laden method into objects that had well defined responsiblities and that encapsulated the high-level policy (heuristic/algorithm/rules).

To give an example the code has 4 variations of calculating a value based ultimately on what is in the DB and the state of a policy (medical insurance policy). The code originally used many conditionals to determine what the DB says the rules are for a given state. These different strategies were interwoven in one method so the high level policy/heuristic/rule whatever you want to call it could not be easily understood.

The end result was that we refactored the code into 1 high level interface, 4 strategies, with 1 strategy being unique and the other 3 having a inheritance relationship (not only for sharing of behaviour but also sharing of implementation).
The sharing of implementation was a by-product of the refactoring exercise and although not ideal the main concern was that we had a well typed polymorphic hierarchy and as a by-product an implementation sharing hierarchy (ala some implementations of Smalltalk collection libraries where the main concern is subclassing for implementation).
The one complex method became 3 lines of code, and the rules were spread over 3 objects. Almost all the conditionals were removed in favour of polymorphic calls and for me this became the crux of the refactoring: Everything seemed more simple and easy to change and maybe even faster (polymorphism-at-its-best)

First post

2008-06-05T16:51:00.000+02:00

Hi

Planning on blogging about IT and software engineering. The blog title is 'stolen' from Alan Kay, he said something to that affect.

Anyway my interests revolve around programming languages (Smalltalk, Java) and all the technology alphabet out there.

Cheers
Carlo