The HP Deskjet and the Mystery of the Blank Pages.

Hello everyone. Time to vent some recent mindfulness. :-)

Remember that HP Photosmart driver I uploaded to OS4Depot years ago? It must have been there for a while now. In the beginning I only uploaded it as a temporary work around. At the time, there was a bug in the OS4 printer drivers, where the C compiled PowerPC code trashed a printer command code double buffer for some reason. On 68K it was fine apparently. This was worked around by changing it a triple buffer. Yet still, it was missed in the OS4 release of the time, so it would be some time before there was an update fix. However, the main source was available of the HP driver, as an example printer driver source code with the fix in place.

I had just bought a new HP Photosmart C4180 for use with document printing and business use. So was keen to have a working printer driver. And not rely on saving out documents as RTF in order to fidget with cables and print them on my iBook. So I chased down the source. Configured it for 600 DPI max density in both B&W and colour which my printer supported. Compiled it. Tested it as working. And then was happy to share the result with other OS4 users. I also wanted to give it a name. The example source was for a HP 1120C. But to me that was little too cryptic and I wanted a more general and friendly name. So I looked at my printer. Ah there we go. HP Photosmart. And so the HP Photosmart printer driver was born. :-)

Somehow, the driver has lasted. Although it was conceived of as a temporary fix, as things in history can go sometimes, it has lasted longer than it was meant too. I mean, it's been years now since all the OS4 HP printer drivers work as they should, on the printers they are specified for. And yet, when ever there talk of someone setting up a HP printer on a forum, in almost all cases the HP Photosmart driver on OS4Depot gets a mention. Now, even though I didn't write the driver code in the first place, it makes me happy to see it mentioned as something little I did helps others in the OS4 community. Perhaps I should go into marketing. LOL! :-D

Now that's not the end of the story. Since the driver did not remain static and did evolve. It was brought to my attention and I had noticed it as well that after a printout the printer would hang for some while, as if waiting for more data that was never to come. Xenic researched this and found that modern printers expected the print dump to be "partitioned" so to speak, by being encapsulated in a print job. Now a print job has job start codes, the main dump, then a job end code. Sounds logical. These other codes are in a language of their own. Called UEL. For Universal Exit Language. It pretty much comes down to an escape sequence of "%12345". To get attention one can send "%12345@PJL". PJL means Printer Job Language.

Now the the OS resource in charge of printing is a device driver by the name of printer.device. Similar to a function library, devices are opened, functions called, and then that device closed. With the added feature that devices have a few device specific functions and specialised commands for block reading and writing. Now with this methodology in place, you'd think it would be simple enough to encapsulate a print dump inside some job start and end commands, since it's opened, used, then closed. Not so simple, as it turns out. I found that a program can open the device, query the printer, then close it. So detecting this from the device open and closing wouldn't always work. Inside the driver, there are routines that deal with bitmap print dumps, where a raster dump is sent to the printer. Even though this is also organised in a similar way I had found; with initialisation, main usage, then freeing of resources it could not be relied upon. Because a program could dump a raster in sections, thus causing repeated use of all the routines. This would mean start and end commands would be sent between each section dump, which wouldn't be correct.

Obviously, I found a solution. And this was, after detecting a job start, to count actual lines printed. Including actual text dumps sent. I had to separate the two as initially one was detected and the other wasn't. Then on driver close check the line counter and if lines had been printed then send a job end. And I was happy to have solved that one. :-)

Time passes. And I had an inkling time would catch up with my driver. It was when my printer ran out of black ink yet again and the colour cartridge was drying up. Funny how I desired a colour printer for years and when I get one hardly print any colour documents or pictures. As is typical with consumer goods, especially HP printers; the older your printer, the more cost it seems to buy a replacement cartridge. And for an official ageing HP cartridge sitting on the shelf from a supermarket to a bulk store, a cost going through the roof. I decided, even though I still liked my HP Photosmart printer, that I would just go out and buy a new one. It would be cheaper than a new set of cartridges. This time a budget HP Deskjet 3630. And soon, I would encounter the typical buyers regret, yet on several different levels.

The mystery? I hooked up the printer to my AmigaOne and set about printing a test page. I was soon met with an immediate problem. The paper was sucked in, the data sent, but after it spewed it back out as a blank page. I also saw after experimentation a rogue "9" character appearing in printouts. I suspected the new printers were more strict with command sequences than previously and later confirmed this. The HP driver is built with PCL3 in mind. That is an invention of HP, Printer Command Language, version 3. Just to throw another acronym in the mix, PCL is considered to be a PDL, a Page description language As stated in technical documentation for common HP printers, ones built in recent years use a variation called PCL3GUI. From research, it means what it looks like when expanded; Printer Command Language, [version] 3, Graphic User Interface. Now historically HP documented PCL versions and command codes. However, for PCL3GUI, this is different. HP does not officially document it. Obviously based on PCL3, but using a limited set of compatible command codes. And for other important operations, such as graphics, a different set of command codes.

I did a little research and put it off until I had time to go into it more fully. It was when I got an email from one Achim, that I knew the problem had spread. Not wanting to leave fans in the lurch, of the HP Photosmart driver that is, ;-) I set about to solve the mystery of the blank pages. And what this PCL3GUI language interpreter expects. I found that this PCL3GUI uses another acronym, as the world likes, called RTL for Raster Transfer Language. This intern, makes use of, yes you guessed it, another acronym, called CRD. For Configure Raster Data. CRD is a PCL command for configuring the printer to receive raster data. It goes into detail describing this in specially formatted binary data structures. This in itself is not unusual, since there are commands to transfer raster data, where the data is binary. Here, in the CRD, it describes things like a DPI width and height matrix per colour pens and describing raster format data. For example, a raster can be a bitmap plane formatted exactly as the Amiga used on the hardware, such as in a black printout. It can be different pens, or even RGB values, where each pen is laid out with its own plane. Or even RGB data in standard chunky format. Had the Amiga hardware been developed beyond AGA, or even developed to where it should have been at that stage, it's likely it would have included these hybrid planar and chunky modes in the hardware. And another thing, which is interesting to note, the byte data in the CRD command format is in big endian order! :-D

So, at this stage, research had led me to forums with people asking what this PCLGUI is and codes for it. There are also sites describing CRD in limited capacity and possibly derived from reverse engineering a PCL dump. As well as examples of PCL interpreters decoding a limited subset of the PCL3GUI CRD command. I also discovered along with this some source code from some HP drivers such as CUPS. And what amused me most is that HP also offer sources for PCL3GUI drivers them selves. Of course there's no real documentation for CRD, apart from some commentary, but there is the source. Use the source Luke! :-P

With this information at hand I set out to update the driver. I needed to examine the output of the printer codes to make sure they were correct. I found I could use the printtofile.device as a substitute for the printer.device. As things are with AmigaOS, it's a little awkward to select this when I needed it, and it seems AmigaOS is made for programmers and not users at times. I went through a few runs until bugs were corrected and the output stream was how it should be. Then it was time for a test print out. As expected, it took a few goes before the printer accepted the codes sent to it, still printing out blank pages. One of the few situations where a blank page wasted my time and a waste of ink was beneficial. I eventually found the sweet spot with codes where the printer accepted the page and what. I also found it was a bit sensitive as to what it did accept as it would ignore certain commands and only accepted a limited amount of parameters.

Happy with the result that it was working with printers again I decided it was ready to share the results. And so I have. Below is the link to download it from OS4Depot. Well thanks for reading. And maybe you can do some printing. :-D

HP Photosmart driver.