I have run maven a couple of times but don’t use it.  (I had to build some open source jars).  And today I decided to clean up my harddrive on my Mac using OmniDiskSweeper.  Low and behold maven created a hidden directory in my home directory called .m2.  In that was 320MB of jar files and classes.  Why does maven need to hide that?  Shouldn’t it be nice and put it in a location that I know about so I can manage it. Instead of hiding junk on me.

I ran across a newer compiled Java language, JavaFX. Check out:

Developing Your First Compiled JavaFX Script Program
— The next JavaFX Puzzler will be posted at 18:00 GMT (1:00PM EST) on Wednesday, November 28, 2007. It will be a compiled JavaFX Script puzzler, so you’ll need to build the JavaFX Script Compiler in order to participate in the Puzzler. To help prepare you for the Puzzler, I’d like to you to develop a very basic ‘Hello World’ style program.

This is interesting since they decided to write their own command line compiler instead of just sticking with Ant Tasks. Looks just like another pre-processor for Java code.

Still don’t understand how the map files that relgen uses is related to CVS. I was looking at the CVS tags and I can see the ones for major release out there, ie R_3_3_1_1 and ones based on dates. But the map files are referring to other version tags such as the date tags. I looks like its a pick and hope type of layout for the map file when choosing versions of dependent source to pull from CVS.

Relgen has a buildAll.xml file that I am going to try to run. Any takers on wether its going to work?

I started trying to build Eclipse today. I knew it would be a big project. As I was suspecting, finding where to start is hard. I know I need to check out the source but how. Looks like you need to use .map files that then reference the appropriate CVS tag and repositories.

I am trying to do this from the command line so I can automate the process. The goal I am trying to get to is a step by step process of building Eclipse not a bunch of pasted together processes from each project. Looks like there is no one way of pulling all the pieces together.

Stay tuned to see if I actuallt get the source checked out.

It appears that the libapr is using the WSARecv, WSASend etc functions instead of the recv and send functions. Also, it setup the connection a little differently than I was. But it was very easy to use. I need to test on Unix but I don’t suspect too many problems.

Check it out at: http://apr.apache.org/

Here’s the step to handle reading chunked data.

1. Read the http headers from the socket looking for the Transfer-Encoding: chunked header
2. After the headers, there will be a new line chr(13) and then a line with only a single number on it. This number is the length of the next chunk of data in hex (use strtol()) to change the hex to decimal)
3. Read from the socket the number of bytes in step #2
4. Next is a possible new line chr(13) or not (depending upon the server) and then the hex number on the next line.
5. You then repeat step #3 until the hex number is 0

Make sure you test against several different servers/clients since the format (extra newlines or not) are not consistent between the servers.

Also, look under the covers to see whats happening. For example, the WebSphere deploy script is really a shell/batch script that wrappers a call to a Java class. Inside the script you can get a better idea of what it expects as inputs and outputs.

The compilers will sometimes use a response file in which all of the inputs and parameters are put into and then the response file is passed to the compiler. Compilers that use response files are great since they avoid the problem of limited command line character limits.

So to get your Windows exes signed you need a certificate from Thawte, Verisign or a like. We have a certificate that we use for signing our jar files that run via a Java Applet. (Only way to make a Java Applet talk to multiple machines and the file system).

The certificate is stored in the key3.db and cert7.db Netscape data files. The trick was to get the certificate from there into something Microsoft would recognize, since you need to use the Microsoft signing tools instead of the Netscape ones to sign a Windows exe.

Here are the steps I did to get me over to a Microsoft certificate store

1. Download the Netscape tools from:
http://www.mozilla.org/projects/security/pki/nss/tools/index.html

An awesome set of tools and documentation for Cryto stuff!!!

2. Since my cert was stuck in a cert7.db instead of a cert8.db I had to go back and get the version of the tools to handle that.
I used the nss-3.6.1.zip and nspr-4.2.2.zip pre-built packages for Windows.

3. I used pk12util to pull the certificate out of the cert7.db

It prompts you for the password to the DB and then another password for the .p12 file

4. At this point I had the certificate in the PKCS12 format in the mycert.p12. Next was to add it to the Microsoft certificate store.

Just run certmgr.exe and a GUI will walk through getting the certificate imported. I just took the defaults. Used my password that
I saved the .p12 file with and Marked the private key as exportable. I left enable strong private key encrytion off. Took the default
of Automatically picking the store. And your done.

5. Now that its in the Microsoft store you can sign your exes. You need to use the signcode.exe or signtool.exe (depending on the version of the Microsoft SDK) to do the signing.

Note: Use a timestamp server /t option

6. To check that it was signed correctly, run signtool.exe verify or chktrust.exe

Thats it!!

It appears that the first fgets() would get all of data except the EOF. The next time around fgets() should return NULL since EOF would be read. But what happens is that the program uses 99% CPU and gets stuck there. I didn’t go into the debugger to see exactly what was going on but the fix was to allocate the buffer bigger than the file size. And use that larger size on the fgets().

Fixed Code

The Virtual Build System that they setup was to have many VMWare images running and have each image run a subset of the build. I theory it looks like it would work well. Each VMWare image dedicated to running a slice of the whole build, so the compiles should run faster. But this is not that case, VMWare actually is getting in the way. No, matter how well VMWare works there are 2 things getting in the way. First, the I/O (network and disk). Secondly, the number of machine instructions executed is high. When the compiler runs, VMWare is running the compiler instructions and passing the instructions on to the Real OS. For VMWare to do this the processor must execute the instructions for VMWare itself as well as the compiler instructions. Therefor, no matter what you do the build is going to be slower.

The Virtualization does buy you a level of abstraction to move a build from one machine to another. But this begs the question, why is the build machine dependent? Shouldn’t a build be able to be recreated on any machine at any time? Openmake was designed with that requirement in place. To repeat a build on another machine you need a OS to run on. Does it need to be the same OS?, no. Running a build on Windows 2000 and Windows XP will produce the same resullts if the compiler version and source code are the same. The compiler can be checked in with the source and checked out with the source if you choose to do so to get a completely traceable build.

Most shops have a standard that they install for the compiler so checking it in and out is usually not need. I recommend just putting the CD on a shelf or off to Off-site storage for disaster recover and archival purposes.

I will get to the first bottle neck and solutions in a future post.

BTW, the Build Meister did confirm that Virtual Build System was slower in the testing that they have been doing. I just reconfirmed their findings.