I fixed a few issues regarding doing thread-unsafe things.  I also put in functionality for your compilation defaults to load from a file.  Disabled the use of multiple tabs (1 FCSH per instance of the app).

Github project URL: http://github.com/EvanMGates/Flex-FCSH-Frontend

SOURCE JAR

Unzip source with 7-Zip, run jar with java -jar FastFlexCompile.jar.

So if you’ve been working on large-scale projects in Flex (especially if they’re monolithic beasts), you have undoubtably noticed how slow the compiler is in Flex Builder 3. This is because (from what I could find) Flex Builder 3 doesn’t allow incremental compilations, which drastically speeds up compiling.

FCSH, which comes with the Flex SDK, does. The problem with FCSH is it’s a pain in the ass to get setup and when you do every time you want to recompile you have to type “compile {target number},” it does not support the up arrow key for last command.

To make things a bit easier and to add highlighting, which I find useful, I made a quick Java Swing frontend to FCSH so you can enjoy quick compilation without dealing with FCSH directly.

To start: Run the app by using the command:

java -jar FlexFastCompile.jar

Put in your SDK path using forward slashes for a Windows machine, making sure to omit the trailing “/”
e.g.

C:/Development/Flex Builder 3/sdks/3.5.0

Put in your mxmlc command line arguments. If you’re unfamiliar with the switches, check out the documentation here. The common ones are: -o (output swf), -file-specs=(application file), -libarary-path+=(library path) for library directories, and -sp+=(path) for source paths.

Click “Spawn New Compiler.” This will execute FCSH and hold the session open in the tabbed window below. To recompile that instance, hit the “Recompile” button.

You can change your sdk and arguments and open up multiple FCSH sessions in the tabbed windows. Be sure to watch your memory usage as too many FCSH sessions can eat up your RAM right quick.

Developer Notes

The GUI and class skeletons was thrown together in NetBeans using Swing components.  The projects was then transferred into Eclipse because I’m more familiar with it.  There are probably still some artifacts from NetBeans in the source.

So the implementation ain’t pretty, and I’m actually passing the Swing JTextArea component down to the output reader threads directly (didn’t feel like messing with thread management much for such a quick and dirty project).  To avoid the possibility of race conditions, I’m using the JTextArea as a lock for the synchronized block that handles the highlighting (:cringe:).  If you want to customize this code for further development, that would be the first place I’d start.

To change the highlighting just edit the wordToLinePainter map in the FCSHOutputDump class. Changing the colors can be done in FCSHOutputDump by editing the DefaultHighlightPainter instantiations.

Feel free to use the code for whateverthehell you want.  There’s no license on here and no warranties as well.  I ask if you do use or modify the code shoot me an e-mail or comment (I like to know if projects are useful or not).  Thanks.

View the app and participate in the network.

It doesn’t really have a name, but for the purposes of this post I’ll refer to it as DDFPOC (Decentralized Distributed Flash Proof-Of-Concept).

DDFPOC is a prototype for a new way of utilizing web traffic for computational purposes. Inspired by projects like BOINC and Folding@Home, I wanted to find a way to have casual web surfers participate in a computational network just by visiting a webpage. Deviating from the BOINC and Folding@Home models and my prior Flash-based distributed computing POC, I also wanted information sharing across the clients as opposed to a central location that aggregates all the separate work units together. Finally, I wanted to have a visual representation of the network so the users could see what’s happening within the network and what the work units were doing.

Implementation Details

The client-side application was programmed entirely in Flex and uses the awesome dependency graph provided with Flare.  The server-side code is in PHP5.  The Flash application makes framework and worker requests to the server in set intervals.  The framework requests are to get the status of the network — who is currently in the network, who’s connected to who, and statistics.  The worker messages are for sending and receiving messages about the current worker.

Spoiler: it’s not really decentralized. I didn’t make the program to do Flash-to-Flash communication, so messages passed between clients are routed through the server.

Green is a message in, red is that same message out.

There are 2 main parts to both the Flex and the PHP: the framework and the worker.  For Flex, the worker module only has to implement an interface.  If this project continues forward (which I doubt due to my project ADD), I would like to be able to add and remove worker modules during runtime.  That way the client could choose which project they would like to donate their CPU cycles to (much like BOINC).  The PHP is a bit more messy, and there’s a couple places where the implementation of the worker is coupled with the user management (such as clearing out expired users).  Aside from these few places, though, the PHP worker is pretty encapsulated and I don’t think it would be much work to make it entirely so.

What’s With The Number Guesser?

I chose to implement a random number guesser as the first POC worker for the following reasons: it’s easy to understand, the knowledge sharing is trivial, it didn’t have to be very visual.

On the PHP side: if there is no work unit out (a number had recently been found), the PHP worker generates a new number between 1-1000 and informs the client that there’s a new work unit.

On the Flash side: keep guessing numbers between 1-1000 until you find the answer or someone else does.

The knowledge sharing works as follows:  keep track of the numbers you guess and tell the neighbors you’re linked to what numbers you’ve tried.  When a neighbor informs you of their guesses, add them to your “already guessed” pile.  Forward on to your neighbor’s guesses to your neighbors.  Eventually the network will guess the number using all of the clients’ memory of guessed numbers.

In The Future

This project was made to be throw-away, but if it were to continue forward I’d first like to completely decouple the framework and worker on the server side.  Next would be making the Flash worker load dynamically.

What I’d love to see is an actual application for this.

Notes

Keep in mind this is a POC and isn’t well tested, I expect bugs .

Also, if there’s any interest I’d be happy to release the source code.

This is the very first step in the processes.  It’s not quite a “framework,” but more of a template.  Click to activate the ants, slide to change the speed.

You can right click and view source.  The code is very rudimentary and the only slightly interesting part is how the ants decide to move.

var stuckCounter:int = 90;

do
{
	var randomAngle:uint = rotation + ((Math.random() * stuckCounter) - (stuckCounter / 2));
	var randomDistance:uint = Math.random() * 70 + 10;

	var randX:int = randomDistance * Math.cos(randomAngle * (Ant.TO_RADIANS));
	var randY:int = randomDistance * Math.sin(randomAngle * (Ant.TO_RADIANS));
	stuckCounter++;
} while (environment.isObstructionInPath(x, y, x + randX, y + randY));

The code is pretty self explanatory (hopefully), but basically the ant is choosing a random direction within its scope (45 degrees both left and right of it’s current rotation), generates a random distance to travel, and checks with the environment to make sure there’s no obstructions within its projected trajectory. If there’s an obstruction, add 1 degree to the randomizer and repeat. Using randomly chosen angles along with a slow increase in the randomized range helps give a more realistic look of obstacle avoidance than other methods (say, flipping rotation).

I’ll be adding more to this section when I get some motivated down time to play, but this week it’s all about the R&R.

Well, for whatever reason, I thought of it a couple days ago and took a day off doing actual work to play with ACM in Flex.

Quick concept rundown.  So basically (from what I remember, don’t quote me) the adaptive culture model describes how information is spread throughout a population, specifically beneficial traits.  I guess an example of this would be success and proximity.  You notice a neighbor being successful, so you incorporate some of their traits into your life in hopes of you being more successful.  This goes from neighbor to neighbor until the successful traits are assimulated into the whole population.  Geography is just an example.  This can also model information spreading through virtual networks and other such fun stuff.

I implemented a quick and dirty presentation of this concept in Flex.  Success is rated by closeness to the target color you choose.  When you click on a couple squares it randomly checks a neighbor to see if it is more successful.  If it is, the neighbor assimilates one of its neighbors traits, which in this case is randomly either the red, green, or blue component.  If the square updates it notifies its neighbors of the change and they go through the same process.

You can change the color while the program is running if you feel like it.  There’s a lot more stuff that’s easily configurable but I got bored of this and am setting it aside for a while.  I’ll release the source when I come back to it later.

It might take a few clicks for the process to start.  Just select your color and keep clicking; when the queue starts filling up you’ll know.

So it sends a request for a “work unit” on a different server I have running a PHP script that builds some simple XML that looks something like:

<assignment>
<type>MD5</type>
<hash>6ded1c8c8b157a391e618ca39eb11e39</hash>
<prefix>Ab$x</prefix>
</assignment>

The program grinds through building MD5 hashes using the prefix and the next 2 characters.  Since this was just a test I’m only testing 126 characters so each work unit generates 15876 MD5 hashes and compares it to the target hash defined in the XML.

I made it so it’ll never find the hash because I wanted to test full work unit work so after it failed to find a match it returns its results to the server and requests a new assignment from the server.

Rinse.  Repeat.

So I got 57 hits during the testing phase with a return of 1128 work units.  This is roughly 18 million MD5 hashes tested.

I’m probably not going to pursue the MD5 collision thing (there’s much better ways of finding collisions instead of just brute forcing it anyway), but it’s an example of how hidden Flash applications can be used to put your traffic to work.

5:30 AM right now and still on the grind >.<  If C# would fucking support SOCKS proxies I’d probably be in bed right now.

So I’m building an HTTP proxy bridge that will be run locally and cycle through proxy lists, that way C# (or any other language that doesn’t support SOCKS) can use the local HTTP proxy and still get the benefits of other proxies.  If anyone is interested e-mail me and I’ll throw over the program and API specs probably.

Edit:

Went home at 6:30 AM.  Certainly can’t complain about a morning like this though:

This was compiled with Dev-C++ but should be find under Linux if you change the system(“pause”);. Oh and the filepath, of course. WordPress wreaked havoc on the code hopefully it works (I doubt anyone will use it anyway this is mainly for my own personal giggles).

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

//Constants
const int NUM_USERS = 480189;
const int NUM_MOVIES = 17770;
const int HASH_MOD = 500000;

/*** training grid
*  Matrix is sparsely populated and will get stack overflows if I try to fill
*  it with a static array.  Use a doubly linked list for rows and columns (movies
*  and users, respectively).
*/
typedef struct rating_node
{
   int movieid, userid;
   char rating;  //char to save space, static cast
   struct rating_node *next_movie;  //This traverses the row, goes to next user in rating matrix
   struct rating_node *next_user;  //This traverses the column, use this for going down movie list
} rating;

//for user hash table
typedef struct hash_node
{
   int userid;
   struct hash_node *next_uid;  //This traverses the hash nodes, use this for finding correct userid
   struct rating_node *user_col;  //This is a pointer to the head node for the user column
} hash;

//Prototypes

//Load Netflix data into arrays
void buildMatrixFromData(rating **movies, hash **users, char *trainingSetPath); //where movies and users are the head node arrays
//Training algorithm
static inline void train_set(int user, int movie, double rating);
//Add a node to the movie row
static inline void addToMovieRow(rating **head, rating *newnode);
//Add a new user on user column
void addToUserCol(hash ***head, rating *newnode);

//Debugging prototypes
void printMovieRow(rating *headnode);
void printUserColumn(hash **headnode, int userid);
void printNode(rating *node);

int main()
{
    //build the head node vectors
    rating *movies[NUM_MOVIES];  //static array of linked list elements
    hash *users[HASH_MOD];  //This is a hash table with linked heads that go to the linked list user columns.
    //Note:  I didn't check for relative primes (I'm pissed enough about recoding but performance was terrible).

    buildMatrixFromData(movies, users, "C:\netflix\training_set\training_set");

    system("pause");
}

void buildMatrixFromData(rating **movies, hash **users, char *trainingSetPath)
{
    int mid = 1;
    int uid, rat, idx;

    for(mid = 1; mid < NUM_MOVIES; mid++)
    {
        FILE *fp;

        char filename[strlen(trainingSetPath) + 15];
        char num[12];
        strcpy(filename, trainingSetPath);
        sprintf(num, "mv_%07d.txt", mid);
        strcat(filename, num);

        fp = fopen(filename, "r");

        //the first line is junk, skip it
        fscanf(fp, "%d:", &uid);

        while(fscanf(fp, "%d,%d,%*s", &uid, &rat) != EOF)
        {
          //Create new node for rating
          rating *newrat;
          if ((newrat = (rating*)malloc(sizeof(*newrat))) == NULL)
          {
             exit(EXIT_FAILURE);
          }
          newrat->movieid = mid;
          newrat->rating = rat;
          newrat->userid = uid;
          newrat->next_movie = newrat->next_user = NULL;

          addToMovieRow(&movies[mid-1], newrat);

          addToUserCol(&users, newrat);
        }
        fclose(fp);

        //For testing movie vector
        //printMovieRow(movies[mid-1]);
    }

    //For testing hash table
    //printUserColumn(users, 1806515);
}

static inline void addToMovieRow(rating **head, rating *newnode)
{
     //base case
     if(head == NULL)
     {
             *head = newnode;
     }
     else
     {
         newnode->next_user = *head;
         *head = newnode;
     }
}

void addToUserCol(hash ***head, rating *newnode)
{
     //use hash
     hash **hashhead = &((*head)[newnode->userid % HASH_MOD]);
     hash *tmp = (*head)[newnode->userid % HASH_MOD];
     //base case
     if(tmp == NULL)
     {
         //Create the new hashnode
         hash *newhash = (hash*)malloc(sizeof(*newhash));
         newhash->next_uid = NULL;
         newhash->user_col = newnode;
         newhash->userid = newnode->userid;
         *hashhead = newhash;
     }
     else
     {
         while(tmp != NULL)
         {
             if(tmp->userid == newnode->userid)
             {
                 newnode->next_movie = tmp->user_col;
                 tmp->user_col = newnode;
                 return;
             }
             tmp = tmp->next_uid;
         }
         //userid not in hash list
         hash *newhash = (hash*)malloc(sizeof(*newhash));
         newhash->next_uid = *hashhead;
         newhash->user_col = newnode;
         newhash->userid = newnode->userid;
         *hashhead = newhash;
     }
}

void printMovieRow(rating *headnode)
{
     if(headnode == NULL)
     {
             printf("Head node empty!n");
     }
     else
     {
         rating *tmp = headnode;
         while(tmp != NULL)
         {
             printNode(tmp);
             tmp = tmp->next_user;
         }
     }
}

void printUserColumn(hash **headnode, int userid)
{
     hash *head = headnode[userid % HASH_MOD];
     if(head == NULL)
     {
             printf("Head node empty!n");
     }
     else
     {
         hash *tmp = head;
         while(tmp != NULL)
         {
             if(tmp->userid == userid)
             {
                //Found our head, traverse down it printing nodes
                rating *rat = tmp->user_col;
                while(rat != NULL)
                {
                    printNode(rat);
                    rat = rat->next_movie;
                }
             }
             tmp = tmp->next_uid;
         }
     }
}

void printNode(rating *node)
{
     printf("mid: %d,  rating: %d,  uid: %dn", node->movieid, node->rating, node->userid);
}

Note on the data structures: You can basically think of this as a matrix. The rows are the movies and you can access the linked list of users through the static array “movies.” The columns require a bit of work. For speed purposes (and because they’re not sequential) they’re stored as a hash table. To get to a specific user, mod the user’s id by HASH_MOD (this value is fairly arbitrary, I just went for a high number so there should be minimal collisions). This will get you the head node of the hash linked list. Traverse through using next_uid checking the userid in the hash node until you find yours. The pointer “user_col” points to the head rating node for that user.

Just a skeleton, if anyone has any other ideas or suggestions be sure to hit me up, this was just a quick, whirlwind approach I had.

Here’s a simple guide to get Debian up on your IBM JS20 using CD as a medium:

Download the PowerPC architecture ISO from Debian’s site. I’d recommend getting the netinst just for getting the base system up.

Burn the CD and put it in the JS20′s media bay. Push the CD button on the blade you’re installing to (it should light up green).

Go into your management module and set the boot sequence of that blade to CD-ROM (under Blade Tasks -> Configuration).

Reboot the blade and start up a rconsole, you can do this by:

sudo rpower -n node reboot && sudo roconsole -n node -c -t &

When you get the boot: screen, use the 64-bit kernel by typing in “install64″ or “expert64″, depending on how much you’d like to customize the installation.

Go through the installation process as normal. Make SURE you do not change the /dev/hda1 partition. It should be about 8.2 megs and formatted as a PReP boot.

After your installation, the CD will pop out of the tray and will reboot. This isn’t good for us since we need the CD for the next step and so you’ll have to physically put it back it.

Boot off the CD once more, but load up the kernel “rescue64″. Go through the steps and spawn a shell on your root partition.

The reason why you’re using the rescue CD is because inittab will try to spawn getty on virtual terminals instead of SOL’s interface. Edit the /etc/inittab in your favorite editor and add the line:

hvc0:1235:respawn:/sbin/getty -L 9600 hvc0 vt100

Above the other getty commands (should start with 1, 2, .. 7).

Now go back into the management module, change the boot sequence to Hard Drive 0, and reboot your blade for a fresh install of Debian Etch!

After inserting the blade, either flip up the LED shield and hit the power button or you can do a rpower -n node on and SOL should come up within a couple seconds.