May 27, 2009

Crowd Sourcing

I've been thinking about Crowd sourcing lately (Like Amazon's Mechanical Turk). Seems like an interesting way to deal with tasks that are not easy problems to solve with computers. I know that there are other sites out there, with different models, but still the same basic idea.

From a worker perspective it seems like many of the tasks are VERY low paid, which lends itself to people quickly loosing interest, and therefor not having a very reliable base of workers. So in my mind it seems like most of the current models will fail to be successful in the long run. I've been wondering what it would take to make a thriving community of both workers and requestors that would be fair and equitable for both. The key seems to be pricing, because of the low pricing, it seems that the lowest common denominator of people do the work. By providing a better way to determine costs and payments, you could create a much more equitable system that benefits everyone. Workers get more work that pays more, and requestors get a better quality of workers to complete their tasks. The question is, how does this happen?

It seems that instead of Requestors selecting the price, there needs to be some way of determining cost based on time spent, skillset needed and/or user feedback. There might also be a voting or bidding system that could fulfill some of the same purposes. I'm still trying to come up with a concrete idea . . . we'll see what happens.

April 23, 2009

Graph Visualization/Interaction

I'm been trying to find a good open source C#/.NET library for graph visualization and manipulation (Not charts/graphs, but graphs as in Computer Science graph theory/nodes/edges.) I have come up pretty empty handed. I'm trying to find a way to visualize and interact with a state machine library that I have. So I just need basic nodes and edges, both with labels and color, and automatic layout algorithms. I've looked and looked and come up empty handed, the best I could find were:
  • QuickGraph, which didn't seem to have any useful interactive capabilities, and depended on other libraries/code for displaying graphs.
  • NodeXL which seemed to be primarily excel based (who does that?), but did have a library component, but didn't seem to be able to even label edges of graphs.
  • Netron which seems to have existed once as open source, but is now something else (The website is very unclear about what/if they are selling), there is an older version of the open source code that has lots of components, but it doesn't appear to be geared toward library use.
  • Piccolo2D seemed promising, but also seemed to not support labeling edges, and also seemed more oriented towards alternate UI design than simple graph display and manipulation.
Since I couldn't find any compelling C# solutions, I resorted to the defacto standard for graph visualization: Graphviz. It's been around for ever and does very well at displaying graphs, but the best interface I could find was the command line app dot, which works well, but not ideal for integration into a library. I've written a small very specialized C# library that converts my state machines into simple dot files, and processes them in dot through the command line to image files. It is far from ideal, but the results are pretty good. Unfortunately the options for interacting with the graphs are limited since all I get is an image. Hopefully someone will come up with a decent graph visualization/manipulation library for C# some day. Until then, this will have to do.

P.S. Just in case anyone is wondering, I did consider using Windows Workflow Foundation for our state machines, but they seemed way to heavyweight, and since it sounds like they are rearchitecting the system for version 4.0 (To finally get it right hopefully?). I decided to steer clear.

April 14, 2009

Distributed Computing

For years now I have been running Prime95 on some of my computers. (I'm 256 on the top producers list right now). But I've decided to try branching out. I know SETI@home is one of the older projects and I decided to give it a try. I'm only a few days into it and I only have about 7,000 credits on my account. It is cool that SETI@home has a client application that is able to use the GPU. So on my quad core system, I can have 5 concurrent tasks (1 GPC, and 4 CPU).

Distributed computing has always seemed very interesting to me as a software developer. Unfortunately there are only certain projects and applications that lend themselves to this kind of architecture. Most applications are tightly coupled and even if they can be "distributed", it's likely that the best you'll get is multiple CPU's on the same host computer. Some tasks can be distributed across multiple computers that are co-located and/or have high speed interconnects like most super computers these days. But even many of these tasks require that the systems are the same hardware/OS/etc, and thinking back to my college class on parallel processing, they can be hard to write well (so they run efficiently and correctly). There really are relatively few projects that fit this mold of being able to be widely distributed across different machines that only communicate on an infrequent basis. But it is a really cool concept, that seems to be gaining more traction in the commercial world as well as these kinds of collaborative volunteer projects.

By the way, the best sites I've seen so far for monitoring stats are BOINCstats and Free-DC. They both seem to have a decent set of graphs and data to analyze my progress. But the other stats sites have some cool stuff as well (Stats 'N Stones, Synergy, The Knights, all Project Stats, Combined Statistics). If anyone else has any favorite projects let me know. I probably try branching out in the next few weeks.

UPDATE: I decided to join Einstein@Home as well, but I'm still open to other project suggestions.

February 26, 2009

UML and struggles with Visio

I've started working on a new project and were trying to design everything properly from the ground up. I've got a rough high level system architecture in place, and now I'm trying to do more detailed design of the individual components of the system. I haven't used formal UML much in the past, so I'm trying to force myself to learn and use it for my class documentation. After some rough sketches on paper and whiteboards. I have tried using Visio to create the diagrams. This has not gone well. Visio seems to force a very strict form of UML, if there is some sort of informal notes, or attributes I want to add, it is almost impossible.

Also it seems that everything you add to the diagrams has to be done through the properties dialog boxes. Why can't I just get a box with three sections, and type what I want. That would seem to be so much easier. Maybe the other method formalizes the UML (and maybe it could be used for generating classes?) but it seems like way to much overhead.

I guess I need to look into other ways to do this with Visio, or even look at some other tools for UML. Maybe a good open source tool?

February 6, 2009

Enterprise level Report designer and generator

I'm looking for a reliable and full features report designer and generator (like Crystal Reports.) Open source or free would be nice, but I'm open to commercial applications as well. My short list of requirements are:
  • Works with MySQL
  • Charting/Graphing (Hopefully with something like excels PivotTable feature)
  • User authentication and authorization that ideally integrates with Active Directory
Crystal Reports is the only one that I have heard of, but I've also heard it has it's shortcomings. My searches haven't uncovered anything that really stands out, so I was wondering if anyone is aware of any good solutions.