If you have any questions about ColdFusion and/or statistics, please feel invited to ask me on martijnvanderwoud @ gmail.com. I will provide clear and helpful answers where I can on this weblog. And where my expertise is insufficient I can probably help you find someone who knows!
I think this can be a fun way of ‘forcing’ myself to blog more often. 
Fire away!
Just a quick post to let everybody know that I have a new job. Starting April 1st, I will be working as senior web developer at Kapaza! Kapaza! is the leading classifieds site in Belgium. With over 200.000 unique visitors each day, the scale of this site is definitely going to offer new challenges. I am thrilled to join a highly technical work environment where I can be a hardcore software developer. I cannot wait to dive in!
Of course I am also leaving a wonderful employer. Being a member of the R&D team of Orga-Toolkit was my first bonafide job. I witnessed the company grow from two (my boss and me) to currently thirteen workers. In many aspects, the past four and a half years have been a rollercoaster. I feel incredibly lucky to have had so much opportunity for experimentation, for the chance to learn so many new things, and – most importantly – for the privilige to work with one of the most enthousiastic, talented and wonderfully joyful set of people I’ve ever come across. You will not be forgotten!
A while a ago, I posted a discussion about different ways to represent complex data, and a UDF to sort arrays of structures on multiple keys. After reading this post, Brian Fitzgerald asked me if I knew of any UDF that could provide the same kind of functionality for an array of beans (instantiated CFC’s). This made me realize that my blog post had omitted perhaps the most useful way of representing complex data in ColdFusion (version 6 and up): ColdFusion Components!
This is not the place for an in-depth discussion of CFC’s, but if you are interested I can recommend this introduction by Hal Helms. For me, the essential advantage of CFC’s is, that they provide the possibility to encapsulate both data and functionality inside one object(variable). This makes it possible for CFC’s to represent real-life objects more naturally than any other CF datatype. CFC’s also provide great possibilities for writing re-usable and portable code, because they can contain the logic that is necessary to handle the specific data they are holding.
As for Brian’s question, it inspired me to improve my sorting UDF, so that now it is able to sort arrays of objects, the object being either instantiated CFC’s (beans), or structures. The UDF, together with two usage examples, can be downloaded here. The code of the examples is shown below .
<!--- Example: sorting an array of instantiated CFCs --->
<!--- Note: this templates depends on person.cfc being in the same folder --->
<!--- Include UDF template --->
<cfinclude template="sortArrayOfObjects.cfm">
<cfscript>
//EXAMPLE USAGE:
//an array of 4 persons, with three properties...
variables.persons = arrayNew(1);
variables.person = createObject('component','person');
variables.person.setName('Martijn');
variables.person.setAge(29);
variables.person.setWeight(78);
arrayAppend(variables.persons, variables.person);
variables.person = createObject('component','person');
variables.person.setName('Jelle');
variables.person.setAge(29);
variables.person.setWeight(85);
arrayAppend(variables.persons, variables.person);
variables.person = createObject('component','person');
variables.person.setName('Lammert');
variables.person.setAge(51);
variables.person.setWeight(78);
arrayAppend(variables.persons, variables.person);
variables.person = createObject('component','person');
variables.person.setName('Bas');
variables.person.setAge(51);
variables.person.setWeight(78);
arrayAppend(variables.persons, variables.person);
// specify how to sort:
// first on age - descending
// second on weight - ascending
// third on name - ascending
variables.sortkeys = arrayNew(1);
variables.sortKey = structNew();
variables.sortKey.keyName = "age";
variables.sortKey.sortOrder = "descending";
arrayAppend(variables.sortKeys, variables.sortKey);
variables.sortKey = structNew();
variables.sortKey.keyName = "weight";
variables.sortKey.sortOrder = "ascending";
arrayAppend(variables.sortKeys, variables.sortKey);
variables.sortKey = structNew();
variables.sortKey.keyName = "name";
variables.sortKey.sortOrder = "ascending";
arrayAppend(variables.sortKeys, variables.sortKey);
// do the sorting
variables.persons_sorted = sortArrayOfObjects(
arrayToSort = variables.persons,
sortKeys = variables.sortKeys,
doDuplicate = false,
useGetterMethods = true);
</cfscript>
<h3>Unsorted:</h3>
<cfloop from="1" to="#arrayLen(variables.persons)#" index="variables.i">
<cfset variables.person = persons[variables.i]>
<cfoutput>
<p>
Position: #variables.i# <br />
Age: #variables.person.getAge()# <br />
Weight: #variables.person.getWeight()# <br />
Name: #variables.person.getName()# </br>
</p>
</cfoutput>
</cfloop>
<h3>Sorted:</h3>
<cfloop from="1" to="#arrayLen(variables.persons_sorted)#" index="variables.i">
<cfset variables.person = persons_sorted[variables.i]>
<cfoutput>
<p>
Position: #variables.i# <br />
Age: #variables.person.getAge()# <br />
Weight: #variables.person.getWeight()# <br />
Name: #variables.person.getName()# </br>
</p>
</cfoutput>
</cfloop>
<!--- Example: sorting an array of structures --->
<!--- Include UDF template --->
<cfinclude template="sortArrayOfObjects.cfm">
<cfscript>
// EXAMPLE USAGE:
//an array of 4 persons, with three properties...
variables.persons = arrayNew(1);
variables.person = structNew();
variables.person.name = 'Martijn';
variables.person.age = 29;
variables.person.weight = 78;
arrayAppend(variables.persons, variables.person);
variables.person = structNew();
variables.person.name = 'Jelle';
variables.person.age = 29;
variables.person.weight = 85;
arrayAppend(variables.persons, variables.person);
variables.person = structNew();
variables.person.name = 'Lammert';
variables.person.age = 51;
variables.person.weight = 78;
arrayAppend(variables.persons, variables.person);
variables.person = structNew();
variables.person.name = 'Bas';
variables.person.age = 51;
variables.person.weight = 78;
arrayAppend(variables.persons, variables.person);
// specify how to sort:
// first on age - descending
// second on weight - ascending
// third on name - ascending
variables.sortkeys = arrayNew(1);
variables.sortKey = structNew();
variables.sortKey.keyName = "age";
variables.sortKey.sortOrder = "descending";
arrayAppend(variables.sortKeys, variables.sortKey);
variables.sortKey = structNew();
variables.sortKey.keyName = "weight";
variables.sortKey.sortOrder = "ascending";
arrayAppend(variables.sortKeys, variables.sortKey);
variables.sortKey = structNew();
variables.sortKey.keyName = "name";
variables.sortKey.sortOrder = "ascending";
arrayAppend(variables.sortKeys, variables.sortKey);
// do the sorting
variables.persons_sorted = sortArrayOfObjects(
arrayToSort = variables.persons,
sortKeys = variables.sortKeys,
doDuplicate = false,
useGetterMethods = false);
</cfscript>
<h3>Unsorted:</h3>
<cfdump var="#variables.persons#">
<h3>Sorted:</h3>
<cfdump var="#variables.persons_sorted#">
Today a colleague came to me with a rather curious error. He had made a UDF, containing the following line of code:
ZS = (arguments.rawScore – arguments.mean) / arguments.SD;
All referenced arguments existed (we checked that multiple times ) and – as you may see, the expression structure seems to be valid (the UDF was written in cfscript).
Still, calling the UDF consistently resulted in the following error:
ColdFusion was looking at the following text:
\u2013
After a few minutes of astonishment another colleague suggested to take a good look at the character at the position reported in the error message. This turned out to be the – in the above line. Nothing seemed wrong with it, but replacing it with a – turned out to fix the problem.
Huh? I could not see the difference either, but the original – had been copied and pasted from a MS Word document, and in fact was not a minus but a unicode ‘EN DASH’ (U+2013). This character is not a minus so ColdFusion was correct in reporting the error. Apparently MS Word had figured out that my colleague actually meant to write an ‘EN DASH’ when he pressed the minus key on his keyboard. Just another reason not to use MS Word to wite down code. Not even short snippets of code, as part of taking notes in a discussion!
Today I wrote a little UDF linearly transforms a value from a given scale, to an arbitrary scale with the specified range.
Say what?
Let’s say I have a test for which the maximum possible score is 25, and the minimum possible score is 5. I might want to rescale that score so that is has a minimum of 0 and a maximum of 10. This could be because I have to report the scores of many different test scores, each with their own minimum and maximum, and I want an easy way to compare these scores. This UDF is a simple way to accomplish this. You can download the code here.
<cfscript>
/**
Linearly transforms a value from a scale with a given mininum and maximum,
to a new scale with given minimum and maximum
*
* @param value - Numeric: the value that must be rescaled.
* @param originalMin - Numeric: the minimum of the original, untransformed scale.
* @param originalMax - Numeric: the maximum of the original, untransformed scale.
* @param newMin - Numeric: the desired minimum for the new, transformed scale.
* @param newMax - Numeric: the desired maximum for the new, transformed scale.
* @return - Numeric: the linearly transformed value
* @author Martijn van der Woud (http://martijnvanderwoud.wordpress.com) at Orga-Toolkit (http://www.orga-toolkit.nl)
* @version 1, August 26, 2008
*/
function linearRescale(value, originalMin, originalMax, newMin, newMax) {
return arguments.newMin +
(arguments.newMax - arguments.newMin) *
(arguments.value - arguments.originalMin) /
(arguments.originalMax - arguments.originalMin);
}
<cfscript>
<!--- the properties of the original scale --->
<cfset variables.originalMin = 5>
<cfset variables.originalMax = 25>
<!--- the (desired) properties of the new, transformed scale --->
<cfset variables.newMin = 0>
<cfset variables.newMax = 10>
<!--- The value that must be transformed --->
<cfset variables.value = 20>
<!--- calculate the rescaled value --->
<cfset variables.rescaledValue = linearRescale(
variables.value,
variables.originalMin,
variables.originalMax,
variables.newMin,
variables.newMax)>
<cfoutput>#rescaledValue#</cfoutput>
This week I’ve been looking for a flexible and easy-to-use Content Management System. My wife is starting her own business in interior design and I am helping her with the company website. Of course, being a ColdFusion lover, I started with checking out the available CF-based platforms. There are a few in http://www.riaforge.org (just type the keyword “CMS” in the project search page). Here are the ones I looked at, and why they are not suitable for what I need.
Bytespring CMS:looked nice and simple. You can check out a working demo on http://demo.bytespring.com. Two things though: bytespring does not seem to have very flexible templating capabilities, and it only works if you intstall it directly under the webroot (I’ve tried to change some paths and mappings but the thing is definitely not portable). For me, having to install a model-glue application directly under the webroot is a definite no-go: I refuse to have folders named “model”, “view”, “controller” directly in the webroot of my local development server, since most of my other applications have the same structure so things would get really confusing. So no local testing, which is the main reason I decided against this CMS.
CAM-II-CMS: seems to be just starting out. Nothing that can be installed and used yet, as far as I can tell.
ActiveCharity CMS: also seems to be unfinished, plus targeted at non-profit users (and we want to make some serious money, don’t we Franca )
CF-Nuke CMS: also no download link in riaforge. The “external project link” points to a page that does nog exist (an all-to-familiar blue CF error sits there straring at us). Stripping “index.cfm” from the url returns the correct page, where we can read that the project is in some kind of midlife crisis . I qoute : “With some reservation, I have decided it was appropriate to post the last known release of CFNuke for download. I say reservation, only because I think there is much room for improvement to this version. I have looked over most of the code but still have more to go. I am most concerned about the security of this application. It is my reccomendation that you download this software for learning purposes only. If you wish to use cfnuke it is highly recommended that you add your own parameter cleaning and queryprams to all queries.” Best of luck to Mark who seems like an enthousiastic guy who is going to work hard to improve on things, but I need to have the site up this month.
Katapult – CMS: I actually got this working just by unzipping some files into a subfolder under the webroot. Works like a breeze, but – like bytespring – not very flexible. I need multiple templates that I can choose from to create a new page. In the templates, I want to have control over the rendered HTML, not just a CSS template. Katapult just lets you customize the stylesheets, and the content is submitted as a single block through an FCKEditor. The editor is nice, but it also means that the user still has to worry about the layout within the page and that is precisely the thing a CMS user should not have to worry about, in my opinion.
Yet Another ColdFusion CMS:From the getting started page, I get the impression that this one could be pretty useful for me. Kept coming up with errors when I tried to install it though. Finally got impatient with it, so I never saw this one working. Could be my bad of course.
Farcry:Looked bloated to me: a lot of feautures but I got lost trying to figure out how everything worked. I need something to edit content on a website, not manage enterprise-level publishing workflow.
PHP – CMS Made Simple:So, I finally did it: I installed PHP on top of my laptop’s Apache server. It sits there next to ColdFusion and the two have not been fighting yet! (In fact, .php pages are extremely slow when CF is not running, probably because the Apache server keeps waiting for a response from CF on every request). After installing PHP, I installed CMS Made Simple, and that’s the one we are going to use. It has sohisticated templating abilities, giving designers and developers fine-grained control over both HTML and CSS in the rendered pages. Within pages templates, multiple named blocks of content can be defined, so that the content manager only has to fill the block with the appropiate text or image, without having to worry about the layout within the page. I supports two mechanisms for SES urls, and is easily extendible. Images can be resized, cropped and rotated from within the CMS. In short, I think it’s really good!
Makes me wonder, could this have been done in ColdFusion? I reckon the answer is yes, and probably it would even have been easier to develop something like this in CF. Problem is, far more people and companies are using PHP than CF and it seems – at least when it comes to CMS’s – that the availability of good developers is more important than the superiority of the technology they are using. In my opinion, this would be the main thing Adobe has to worry about when it comes to marketing ColdFusion.
This photo was taken at the Cavern Club in Liverpool, during the ITC 2008 conference. We had a great time, thanks to the Beatles Experience.
My collegues and me with the Beatles experience.
My colleague Jelle Geertsma just did his presentation on the Webcam Test Communication, which is a great new instrument developed by Jelle at Orga-Toolkit, in cooperation with Lonneke de Meijer at Erasmus University, Rotterdam. You can download a copy of his slides here.
Jelle Geertsma, explaining how he developed the Webcam Test Communication, and reporting some promising results.
Today I’ve been presenting at the ITC 2008 conference in Liverpool. I talked about the test-norming methodology research which I’m doing with Peter Tellegen. The presentation went well, thanks to great feedback from my colleagues at Orga Toolkit. The slides of my presentation can be downloaded here.
Explaining the importance of improving methods for test norming.
Today, a collegue asked me to write a UDF to calculate the Pearson correlation. Of course, this kind of assignments are fun for me. After all, I like statistics AND ColdFusion, so what could be better than combining these? OK, that sounds a bit dorky , but it’s always nice to write code for doing calculations that you would not usually do by hand. It makes it easier to understand what is going on when you use software like SPSS to do the calculations for you.
So here is my code for calculating the Pearson correlation. It can also be downloaded HERE, and I will submit this to CFLIB.org as well. Comments and criticism very welcome!
<cfscript>
/**
* Returns the pearson correlation between (the values inside) two keys in an array of structures .
* Values must be numeric, obviously
*
* @param arrayOfStructures - An array of structures containing the specified keys for every element.
* @param xKey - A string: the structKey containing the first variable.
* @param yKey - A string: the structKey containing the second variable.
* @return Struct with two keys:
* pearsonCorrelation: a number ranging from -1 to +1, or an empty string if inputValid is false.
* inputValid - boolean: indicates if the input was valid, so that the pearson correlation could be computed
* (pearson correlation cannot be computed if at least one of the variables has zero variance).
* @author Martijn van der Woud (http://martijnvanderwoud.wordpress.com) at Orga-Toolkit (http://www.orga-toolkit.nl)
* @version 1, July 9, 2008
*/
function pearsonCorrelation (arrayOfStructs, xKey, yKey) {
// numeric: holds the mean value for the xKey
var xMean = 0;
// numercic: holds the mean value for the yKey
var yMean = 0;
// numeric: just a loop index
var i=0;
// numeric: holds the sum of all values for the xKey
var xSum = 0;
// numeric: holds the sum of all values for the yKey
var ySum = 0;
// numeric: the number of elements in arrayOfStructs
var length = arrayLen(arguments.arrayOfStructs);
// numeric: the sum of squared deviations for the xKey
var sqDevX = 0;
// numeric: the sum of squaried deviations for the yKey
var sqDevY = 0;
// numeric: the sum of cross-products
var crossProductSum = 0;
// numeric: holds the deviation from the mean for the xKey in a specific element
var xDeviation = 0;
// numeric: holds the deviation from the mean for the yKey in a specific element
var yDeviation = 0;
// numeric: the Pearson correlation
var pearsonCorrelation = 0;
// struct: the results to return
var results = structNew();
// loop over elements in argument arrayOfStructs
for(i = 1; i lte length; i = i+1) {
// add the xKey and yKey values of the current element to their corresponding sum variable
xSum = xSum + arguments.arrayOfStructs[i][arguments.xKey];
ySum = ySum + arguments.arrayOfStructs[i][arguments.yKey];
} // end of loop over elements in argument arrayOfStructs
// calculate the means of xKey and yKey
xMean = xSum / length;
yMean = ySum / length;
// again, loop over elements in argument arrayOfStructs
for(i = 1; i lte length; i = i+1) {
// calculate deviations from the mean for the current element
xDeviation = arguments.arrayOfStructs[i][arguments.xKey] - xMean;
yDeviation = arguments.arrayOfStructs[i][arguments.yKey] - yMean;
// update sums of squared deviations and cross-products
sqDevX = sqDevX + xDeviation^2;
sqDevY = sqDevY + yDeviation^2;
crossProductSum = crossProductSum + xDeviation * yDeviation;
} // end of loop over elements in argument arrayOfStructs
// if there is no variation in either xKey or yKey, the pearson correlation cannot be computed, so indicate an error
if (min(sqDevX, sqDevY) eq 0) {
results.inputValid = false;
results.pearsonCorrelation = "";
} else { // otherwise, calculatie the pearson correlation
pearsonCorrelation = (crossProductSum / (length-1));
pearsonCorrelation = pearsonCorrelation / sqr(sqDevX / (length-1));
pearsonCorrelation = pearsonCorrelation / sqr(sqDevY / (length-1));
results.inputValid = true;
results.pearsonCorrelation = pearsonCorrelation;
}
return results;
} // end of function pearsonCorrelation()
</cfscript>
<!--- example --->
<!--- An array of structs, with keys "X" and "Y"; all values in X and Y are numeric--->
<cfset variables.arrayOfStructs = arrayNew(1)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 3>
<cfset variables.element.Y = 1>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 6>
<cfset variables.element.Y = 2>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 7>
<cfset variables.element.Y = 3>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 4>
<cfset variables.element.Y = 4>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 5>
<cfset variables.element.Y = 5>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 3>
<cfset variables.element.Y = 6>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 7>
<cfset variables.element.Y = 7>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 6>
<cfset variables.element.Y = 8>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 5>
<cfset variables.element.Y = 9>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 2>
<cfset variables.element.Y = 1>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 3>
<cfset variables.element.Y = 2>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 5>
<cfset variables.element.Y = 3>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 6>
<cfset variables.element.Y = 4>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.element = structNew()>
<cfset variables.element.X = 7>
<cfset variables.element.Y = 5>
<cfset arrayAppend(variables.arrayOfStructs, variables.element)>
<cfset variables.pearsonCorrelation = pearsonCorrelation(
arrayOfStructs = variables.arrayOfStructs,
xKey = "X",
yKey = "Y")>
<cfdump var="#variables.pearsonCorrelation#">
