Dr. Hunter of the University of Colorado wrote a handy implementation of many statistical functions for Common Lisp. The package is called cl-statistics and is available via Unbuntu’s package manager. However, an inverse F cumulative distribution function is mysteriously absent. That is, a function that will provide an F statistic given two degrees of freedom and a percentile. Luckily, it’s pretty strait forward to modify the source to add it. In hopes of saving someone a couple hours of fumbling around, here’s the modification; just add the following function to the cl-statistics source file and add the function name to the export list near the beginning of the file.

(defun f-distribution (dof1 dof2 percentile)
(test-variables (dof1 :posint) (dof2 :posint) (percentile :prob))
(find-critical-value
#’(lambda (x) (f-significance x dof1 dof2 :one-tailed-p))
(- 1 percentile)))

I have long been interested in how the selection and use of a particular programming language by the programmer influences how they represent the problem internally. Ultimately, this would influence how, or even ‘if’, they solve the problem. As part of my studies at the University of Washington, I’ve finally had the opportunity to do some actual research in this area, albeit of very limited scope. The paper below examines the observed differences in the semantic networks of three programmers. An emphasis is placed on differences between users of languages in different paradigms, such as functional versus imperative.

This interest grew back when I was making heavy use of both the functional and interactive features of the Lua language. Over time, programming began to ‘feel’ different and I notices I was using different types of design patterns as compared to my C++ heavy days. These and many other subjective aspects encouraged me to explore other uniquely variant views of software development such as Smalltalk and Qi. I became further convinced the linguistic homogeny pervasive across industry is detrimental to innovation.

Unfortunately, on the topic of ‘what language is suited for what’ practitioners and researchers alike have little more than anecdotes. After all, most general purpose languages are all Turing complete and therefore computationally equivalent, so why does it matter? It matters when one considers the programmer and the machine as a composite unit. Then, the programmer is involved in a stimulus/response loop with the machine where the programming language and environment is the communication medium. At this level, the way the programmer perceives and responds to the machine state becomes important. I assert that language influences this interaction in an under-appreciated fashion regardless of underlying technical equivalencies.

Semantic Organization of Programming Language Constructs (postscript format)

“To be truly challenging, a voyage, like a life, must rest on a firm foundation of financial unrest. Otherwise, you are doomed to a routine traverse, the kind known to yachtsmen who play with their boats at sea… cruising, it is called. Voyaging belongs to seamen, and to the wanderers of the world who cannot, or will not, fit in. If you are contemplating a voyage and you have the means, abandon the venture until your fortunes change. Only then will you know what the sea is all about. “I’ve always wanted to sail to the south seas, but I can’t afford it.” What these men can’t afford is not to go. They are enmeshed in the cancerous discipline of security. And in the worship of security we fling our lives beneath the wheels of routine - and before we know it our lives are gone. What does a man need - really need? A few pounds of food each day, heat and shelter, six feet to lie down in - and some form of working activity that will yield a sense of accomplishment. That’s all - in the material sense, and we know it. But we are brainwashed by our economic system until we end up in a tomb beneath a pyramid of time payments, mortgages, preposterous gadgetry, playthings that divert our attention for the sheer idiocy of the charade. The years thunder by, the dreams of youth grow dim where they lie caked in dust on the shelves of patience. Before we know it, the tomb is sealed. Where, then, lies the answer? In choice. Which shall it be: bankruptcy of purse or bankruptcy of life?” - Sterling Hayden (General Jack D. Ripper from Dr. Stranglove).

It’s official, I’m a student again. For this reason my posting frequency is likely to be spotty for the foreseeable future but I’ll try to keep it up. At any rate, the focus may change a bit as I no longer have enough spare cycles for existential critiques or cowboy programming with neural networks However, I do have a couple term papers to write that should yield some interesting post topics. One paper will likely focus on programming languages or environments from a usability perspective. The other may be on the same or similar topic from a cognitive perspective, such as investigating the mental models and situational awareness of a programmer involved in ‘to be scoped’ tasks.

The notion of an urban village has existed for several decades. In contrast to traditional city planning, an urban village attempts to minimize the daily need for personal cars, long-haul transit for foodstuffs and general dependence on a ‘foreign’ source for daily necessities. Foreign in this context may be interpreted as non-local corporate entity. A casual glance at modern cities in the West on the whole reveals the opposite ideology where most goods are shipped anywhere from hundreds of miles away to the opposite side of the globe. Although this strategy protects some daily needs from the toll of a local disaster, the cost in terms of energy is incalculable. That is, one can not calculate the true cost of natural resources and arguably labor. Traditional city design also incurs social side effects. The consumer is not responsible for, and thus has no direct power, over the production and distribution of daily needs. This has its most obvious impact on the poor who are beholden to the aloof and invisible master that produces and distributes their food and energy.   

Alternatively, an urban village design facilitates some local food production. Certainly, growers will still expect some form of compensation for their crop. However, when production is local, labor may be exchanged with an employer who has a direct interest in the well being of the community as opposed to quarterly profits. In addition, sound planning and policy may allow lower income individuals some direct control over their own necessities. The Greenwood neighborhood of Seattle Washington recently released the following proposition for the city of Seattle.

“…develop new standards or incentive programs that encourage incorporating food gardens into multi-family developments” and “Directing the Department of Neighborhoods to identify the most suitable City-owned properties for conversion to use for food production, asking for no less than two acres that could be developed in 2009-2010″

It remains to be seen if these propositions succeed and the scope at which they’re implemented. However, to see such ideas being seriously considered by a significant mass of the populous fills one with optimism. I will post updates as they unfold.

Last night I watched Fellini’s 8 1/2 for the second time. Although I enjoyed the movie the first time around, I think I was too focused on the dialog, figuratively and literally since I require the subtitles. This time, I ignored much of the dialog and really payed attention to the visual aspects of the film. Fellini, as compared against many modern filmmakers, makes incredible use of as many informational ‘axes’ has he can find in the visual domain. For instance, in one scene the emotional state of the character is described by the movement of his feet (his face is out of frame). A tremendous amount of attention is given to detail that contributes to the overall aesthetic of a scene, thus increasing the amount of information that can be delivered to the viewer; information beyond the dialog and superficial appearances. For example, modern filmmakers appear to have a fetish with presenting reality. However, in doing so they sacrifice additional information bandwidth that can be delivered to the viewer by dispensing with convention. The hyperreal and surreal provide additional dimensions of information that can be delivered in parallel with the ‘real’. Fellini leverages this dimension by experimenting with non-traditional modes of visual exposition. There seems to be intent behind seemingly mundane actions such as the way a woman is running her hands across a sheet or the way a person is kicking their feet while sitting. In addition, he merges fantasy and in-film reality in sometimes difficult to delineate patterns.

My overall impression was that far more information was being delivered to me, due to this efficient use of the artistic space, when watching 8 1/2 then is present in more traditional styles of films. In the traditional filmmaker’s reluctance to ‘overload’ the film-reality with ‘unreal’ or exaggerated elements they place arbitrary limits on the types and amount of information that is conveyed to the viewer.

I began experimenting with neural network code a number of months ago when I decided to implement a network capable of maze navigation. I had read about NNs but I wanted to have the level of understanding that comes with actually implementing such an algorithm. In this post I shall discuss the design of the NN framework, the NN intended to solve mazes, the problems with said design and my thoughts on NNs in general and connectionism. The code was all written in Common Lisp (SBCL) in Eclipse/CUSP on a Linux system.

The maze environment was realized as a 2D array of 0s and 1s where a 1 represented a wall. The mazes were randomly generated using a recursive spatial partitioning algorithm. There are some pretty pictures in the programming category of this blog. The decision to have mazes be the problem domain was somewhat arbitrary and introduced some complexity. In retrospect, finding efficient solutions to mazes was not the best problem to tackle with a simple NN. Although it seems totally doable with the right network design.

The implementation of the NN framework leveraged the Lisp object system allowing for simple extension of the algorithm. The default Neuron object was associated with a set of generic methods Activation, Learning and Propagation. A user wishing to change the Activation function could simply inherit from Neuron and override Activation for their Neuron type. Different types of Neurons and their associated functions could be easily intermixed within the same network without much explicit ‘glue’ code. The implementation also included an object/functional abstraction of a neural layer as used for feedforward networks. The highest level abstraction, feedforward-nn, encapsulated a set of neural layers, an explicit input layer and output layer, and user defined functions that specify the mapping from the problem domain to the input and output layers. At least from my own use, these abstractions facilitated very good flexibility and ease of reuse. They also map back intuitively to typical theoretical specifications of NNs.

I quickly discovered the difficult aspect of NNs is not the implementation of an NN framework but rather the design of a network to solve a particular problem. Initially, I considered ‘evolving’ a network using genetic algorithms. Indeed, this would have been neat especially if it worked. However, I felt that would divert my focus too much and require even more time. I settled on an intuitive design that included 9 input neurons and 8 output neurons; there were no intermediate layers. The neurons were wired in feed-forward fashion with this exception of the output layer feeding back into the input layer (more on this in a moment). The first 8 neurons of the input layer represented a ‘touch sense’ where the neuron was set to fire if a wall was present in that direction (8 possible directions for a 2D cellular environment). The output layer represented the direction to move where the sum of the output neuron’s activation levels was [0, 8]. The 9th neuron on the input layer was a ’smell sense’ which fired if the agent was closer to the cheese/exit than it was on the previous pass. Now, in order to potentiate a ‘correct’ move, a move closer to the cheese, the agent must ‘remember’ the firing sequence of the output neurons from the last pass; hence the aforementioned feedback from the output layer to the input layer.

Unfortunately, this critter never does get to the cheese. Typically after a few moves it settles into a corner and never departs. After a direction is established, the critter becomes more and more determined to keep going that direction. To be successful, there must be some notion of de-potentiation after continued lack of stimulus on neuron 9. That way, it would eventually try another direction even when the input stimulus remains constant. Therefore, I believe some additional tweaking could produce a network capable of solving the mazes.

Regardless, I feel I’ve accomplished what I set out for which was to obtain a fundamental understanding of NNs, their potential and their limitations. Of particular interest to me when I started the project was to evolve my own opinion regarding the biological validity of NNs. I believe a major shortcoming of the current NN definitions, in the scope of biological simulation, is the total reliance on changes in ‘physical’ representation to map to changes in behavior. Given identical input to an NN you will have identical output until the weights, part of the physical representation, are altered. However, I’m skeptical that changes in biological behavior can be modeled purely from these representational changes. There are meta-representational aspects that must be considered such as firing frequency. More simply, you have the wire but you also have what is transmitted ‘over’ the wire. NNs appear to focus on changing the wires to produce changes in behavior. Indeed, it is known that long term potentiation does exist and it likely does produce changes in behavior. However, I assert that is likely not the only mode under which behavioral changes occur and NNs do not currently encompass any mode besides LTP. Rhetorically, how would one describe the behavior of working memory using a model dependent on LTP? To be clear, this is not to say that the necessary behavior can not ‘emerge’ from an NN. I’m merely stating that the current formalism, in of itself, doesn’t seem to allow for such behavior.

Another concern regarding the use of NNs to understand biological function is the potential complexity of the NN resulting from an attempt to model an abstract biological function such as the meaning of an object (I am assuming that the mind is ultimately reducible to the physical or computational realm). As I eluded in the previous paragraph, I assert that abstract concepts are meta-representational, or that one must look beyond a simple neural correlate snapshot to understand how the more complex areas of the mind function. If this is true, then a model at the level of an NN may prove to be as much as a mystery as the physical reality it was intended to model and resist proofs of its correctness.

In the end, NNs will probably continue to prove valuable to those researching at the granular level, neuroscientists, biopsychologists, etc. There is also the professional domain that can apply them to interesting problems regardless of their biological validity. However, I’m skeptical regarding the applicability to the investigation of abstract constructs such as the meaning of objects and reasoning.

I was first introduced to this term when I took a course in psychological research methods a few months ago. Essentially, it means defining a concept in such a way that it can be manipulated in objective space as opposed to subjective space. More simply, a a concept is defined in terms of measurable qualities rather than qualities for which no objective measure exists. Dealing with the objective and things defined operationally is key to scientific method in general and certainly not limited to psychology. Indeed, limiting rationality to the objective space has provided a framework under which great strides in discovery have been and are made feasible.

However, we must take care not to overextend the application of objective reasoning. Today, objectivity is applied almost without question to politics, social sciences and even the meaning of existance. It has been my personal experience that this far reaching application tends to be self-limiting at best and destructive at worst. On the limiting side of the spectrum, we can invoke the Marcusian notion of the operationalization of concepts lke “freedom,” where the objectification of freedom inherently limits what freedom ‘could’ be or mean. On the destructive end of the spectrum, we may postulate that the total absence of subjectivity leads to nihilism in that the limiting factors of objective definition become so pervasive that no room exists for personal discovery or growth.

In the strictest sense, an abstract operational definition would need to be a combination of other operational definitions eventually reducing to a set that are directly measurable. Therefore, a positivist may assert, “even very abstract concepts such as freedom and existence will eventually be explainable through operational definition via reduction of these concepts down to physical or computational laws.” However, the complexity of this model may not be graspable. That is, understanding the model may prove as difficult as understanding the physical manifestation the model was intended to represent. In other words, one would need to apply the same empirical methods to understand the model as they would to understand reality.

Furthermore, in the discourse of social and political sciences definitions are routinely established and used operationally that are not “pure” operational definitions. That is, they don’t reduce to directly measurable elements. This is an almost humorous constraint in these domains because they’re limited by a sense of objectivity which fails to truly be objective.

For better of for worse, objective thinking is so incredibly embedded in our thought processes at this point that it is overextended in attempts to tackle problems for which it is ill prepared. This over dependence opens the door to manipulation and limitations upon thought.

This year, Nov. 4th encompassed an atmosphere more reminiscent of a new year’s eve than an election day. Just as I was finishing the last sip of a tasty 0.5 liter Franziskaner at a neighborhood pub, the broadcast announced Obama as our president elect. The pub patrons howled and cheered as a dark chapter in American history begins to close.

The people finally realize they’ve been manipulated by an administration willing to rule by fear and they don’t want another leader with such ties. On the way, a man has demonstrated the American spirit in action and made skeptics, such as myself, consider a more optimistic attitude with respect to the political system and the people of this land.

As I walked home, there were cheers emitting from the windows of residences, packed bars full of merry making and at least one person running down the sidewalk clanking a ladle against a pot declaring the president elect. I was surprised and taken by the level of positive exuberance which I have never seen before in response to an election. Nov. 4th 2008 will indeed be a night remembered by many.

The blog Neuronarrative, http://neuronarrative.wordpress.com/, has an interesting post on “Identify Politics.” In sum, it’s the notion that politicians will present themselves as characters that a large demographic will relate with and thus will gain their vote. Indeed, this tactic worked well for Bush Jr. by presenting himself as a modest “good’ol boy” rather than the product of unmatched privilege.

I also notice this sort of posturing in the corporate environment, albeit a slightly different manifestation. Anyone wishing to “climb the ladder” must present themselves as being “compatible” with the culture of management in ways that are obviously outside the scope of work such as family, interests, politics and even alcohol tolerance (there are studies that correlate people who drink with co-workers with higher wages).

Ultimately, people make behavioral projections based on how close a perceived identity matches their own. The people you work with extrapolate what you may do in the future by identifying you with some preconceived generalization. In politics, if people feel they identify with the politician they’ll say, “I just know he’s going to do ‘this’ because he/she’s like me.”

When you don’t make an effort to present yourself as “a compatible identity,” and believe me I have some practice at ignoring this aspect of social dynamics, you’re seen at best eccentric or worse a loose cannon. Rather than an asset, you’re a liability because the people essentially “don’t get you” and feel that you’re less predictable than those the person(s) can identify with, regardless of actual behavior.

In day to day life, this particular social dynamic is probably necessary. It is certainly very ingrained in most people. However, politicians, advertisers and anyone else seeking gain can exploit this behavior characteristic without many people being aware of it at a conscious level. I find that dangerous and unethical.