Enigma-free image file

[dilbengineer]

I’m trying to create an Enigma-free image file for the Dreambox.

I have nothing against Enigma or the various other alternatives, but my interest in the Dreambox (DM500S) is just as an embedded Linux box with a DVB frontend and an Ethernet port. No display device or graphical user interface or embedded web browser is desired. This 3+ megabyte software just takes up flash space, CPU time, and makes DVB API calls that could interfere with the lower-level software I’m trying to write.

So far, however, I haven’t had a lot of success, and would be appreciative of any advice. It seems like a development platform like this would be something that someone would have before, but I haven’t found it.

My three avenues thus far have been:
1) Use an existing image (like v108 for DM500S), and kill the Enigma processes (This would only be good for testing; it wouldn’t work in production)
2) Check out the latest CVS files and remove Enigma from the build
3) Take an existing image and edit the SquashFS partition

#1 is a disaster. Enigma does NOT die quietly. If one does a “killall –9 enigma”, it crashes the box, corrupts the flash, and prevents it from working again. There are a lot of nasty messages that get spit out on the serial port before it goes completely dead. To make a box work again after this, one has to totally erase the flash and reload the image via the serial bootloader.

Clearly, this isn’t a viable solution.

For #2, when I found Andreas Monzner’s excellent tutorial for building an image from the CVS files, I was hoping that I had found an easy solution:

http://cvs.tuxbox-cvs.sourceforge.net/l ... 00073.html

The instructions indicate that one can remove “.part_*” files to change the build. There is a .part_enigma file in root/cdkflash, so I tried removing it as described in the tutorial. However, the Makefile puts in Enigma in anyway.

I’m still trying to reverse engineer the whole build process, and figure out how to change it. Does anyone have any insight into how the “.part_enigma” file was supposed to work?

For #3, after a lot of forum searching, I eventually figured out that a Dreambox image is a concatenation of a CramFS partition containing the kernel (and a link to “root”?) and a SquashFS partition containing the root filesystem.

The CramFS portion is the first 1152 * 1024 bytes; the SquashFS is the rest.

I was trying to mount the CramFS portion of the image file as a loop file on a x86 box, and couldn’t understand why it said the magic number was wrong. When I looked at it in a hex editor, it looked just fine to me. Eventually, it dawned on me that the endian-ness was wrong, and wrote a quick and dirty program to swap the bytes around. Duh! Then, it would begin to mount, but it still complained “root is not a directory” and failed to mount it.

However, having access to that portion shouldn’t matter unless the kernel needs to be changed. UNLESS: Does the “root” thing have to be updated to reflect any changes in the SquashFS portion?

For the more relevant SquashFS portion of the image file, it only seems to mount if I don’t change the byte order. However, all the files sizes are wrong. (BTW, trying to open any one of these files (each of which it thinks is several gigabytes in size) in vi is a good way to crash your x86 Linux box! )

Does anyone know what the secret is to mounting a Dreambox SquashFS image under x86 Linux, where it not only mounts the file but actually has the correct file sizes?

Thanks.

[dilbengineer]

OK, I've found a way of creating a new image, but nothing is as simple as it seems.

If one does a complete build using the CDK, the root filesystem will be in ./root/cdkflash/root

The best thing to do is to "cp -a ." to another directory. (If any of the files go missing, the Makefile doesn't like it one bit! ) Plus, having a copy allows one to go back to the original if one messes up somewhere.

One can edit the files (like just deleting ./bin/enigma).

Then:

mksquash <new path of root filesystem> <new squashfs image> -be -all-root

Then, do a binary concatenation of this file to the boot cramfs (see Makefile for a way of doing this with dd).

This all seems to work, and I downloaded the new image.

First off, it is extremely difficult to know what is going on since the serial port isn't the console.

I tried sniffing the network to look for a DHCP request, but nothing is happening.

I don't know if one of the rc scripts is getting stuck, whether the cramfs boot image has to be modified too, or anything else.

Does anyone have any suggestions?

Thanks.

[dilbengineer]

OK, the trick to using the serial port as a console is as follows:

Connect a dumb terminal to the Dreambox serial port. (115200,n,8,1 xon/xoff)

Power cycle the Dreambox.

At the dumb terminal, when one sees the kernel arguments:

console=null root=/dev/mtdblock5 rootfstype=squashfs ro

Hit the backspace key. Continue to backspace, and replace it with:

console=ttyS0,115200 root=/dev/mtdblock5 rootfstype=squashfs ro

and press Enter.

Then, one will see the kernel boot (or not boot).

In my case, I'm getting a kernel panic:

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(31,5)

[dilbengineer]

Here's a variation on the aforementioned avenue #1 that doesn't cause Enigma to go nuts and corrupt the flash:

Load the standard 108 release.

Connect a dumb terminal and change the kernel parms to use the serial port as a console (as described previously).

Enigma preempts the kernel console's use of the serial port, and starts spitting out a bunch of debug messages.

Hit "Cntl-C". This completely kills Enigma! AND, it leaves everything in a normal state (instead of the flash 'scorched earth' policy that happens when you killall).

Hit Enter to enable the console... Presto! You have a Dreambox without Enigma running.

Now, if only I could get an Enigma-free modified image to boot like this without user interaction.

[dilbengineer]

Just an update...

The kernel panic was my own fault. The procedures I described previously were correct; I just made a stupid error.

I’ve since made the Enigma free image. Hooray! I've never seen a Linux box boot and reach a console so quickly.

Things worthy of note:

To be tidy, one should update the init.d scripts to not call the executables that have been removed.

Enigma spawns udhcpc (DHCP client). So, when you remove Enigma, unless you add udhcpc to the init.d scripts, the box won’t initialize the Ethernet interface with an IP address.

Having a working console on the serial port is very useful. (“Don’t leave home without it.”) However, to have this by default requires a recompiled kernel. As part of the Makefile process, a kernel “.config” file, specific to the box version, is downloaded. I have a working procedure at the moment that involves starting anew with a fresh copy of the CDK and modifying the .config file halfway through. However, I’m sure there is a more efficient way.

Also, there are various Enigma-specific files under /var_init (used to initialize /var) that are no longer needed and can be removed.

[cydine]

Having a working console on the serial port is very useful. (“Don’t leave home without it.”) However, to have this by default requires a recompiled kernel. As part of the Makefile process, a kernel “.config” file, specific to the box version, is downloaded. I have a working procedure at the moment that involves starting anew with a fresh copy of the CDK and modifying the .config file halfway through. However, I’m sure there is a more efficient way.

Could you detail the procedure you use at the moment. It would be useful for me also.

Thanks.

[dilbengineer]

No problem.

Disclaimer: I’m certain there is a nicer way, but this worked for me.

I used the standard build process as a basis:

http://cvs.tuxbox-cvs.sourceforge.net/l ... 00073.html

What follows is what I did. It is mostly the same as the link above, but with a few additions. I created a directory to store the CDK, changed to that directory and then:

> export CVS_RSH=ssh
> cvs -d anoncvs@cvs.tuxbox-cvs.sourceforge.net:/cvs/tuxbox -z3 co -P -rdreambox .
> cd cdk
> chmod ugo+x prepare

For a different hardware, the prepare argument is different; my box is a DM500.

> ./prepare dm500
> make checkout

OK, here’s where it gets different. I manually edit ./cdk/Makefile to add "sh" just after the Linux kernel .config file is written.

Note: you might want to check that "sh" is even installed on your particular box, and adapt if not. "csh" or "bash" are perfectly equivalent substitutes (for this task... I don't want to start a flame war about what the best shell is! ).

So, the normal Makefile might look like this:

# cat linux-2.6.9/arch/ppc/configs/dm7000_defconfig > linux-2.6.9/.config
# cat linux-2.6.9/arch/ppc/configs/dm56x0_defconfig > linux-2.6.9/.config
cat linux-2.6.9/arch/ppc/configs/dm500_defconfig > linux-2.6.9/.config
$(MAKE) -C linux-2.6.9 oldconfig \

And I changed it to look like this:

# cat linux-2.6.9/arch/ppc/configs/dm7000_defconfig > linux-2.6.9/.config
# cat linux-2.6.9/arch/ppc/configs/dm56x0_defconfig > linux-2.6.9/.config
cat linux-2.6.9/arch/ppc/configs/dm500_defconfig > linux-2.6.9/.config
sh # very evil: interrupt the make process to allow me to manually change .config
$(MAKE) -C linux-2.6.9 oldconfig \

Then:

> make dreamboximage_root

During the build process, it reaches the sh statement and gives me a shell prompt. I then edit ./cdk/linux-2.6.9/.config to reflect the alternate kernel arguments I want.

The default is this:

CONFIG_CMDLINE="console=null root=/dev/mtdblock5 rootfstype=squashfs ro"

and I changed it to this:

CONFIG_CMDLINE="console=ttyS0,115200 root=/dev/mtdblock5 rootfstype=squashfs ro"

I then:

> exit

to return control back to the Makefile build.

From then on, it is the same as the original build process:

> make rebuild-flash
> make flash-compress

The resultant files should be the same as normal, but the default kernel arguments should now be whatever you set them to.

[cydine]

Can't get much clearer than that

Cheers m8. Much appreciated.

[Barf]

To be able to build an image completely without the GUI paraphernalia is clearly desirable. I therefore wrote some newmake targets flash-null-jffs2-1x and flash-null-jffs2-2x building jffs2-images for this purpose. (Creating cramfs- or squashfs-images for this purpose seems slightly silly.)

Yes, I know that you guys are into Dreambox, and newmake does not support Dreambox (although it would be do-able).

BTW, I have to comment on:

#1 is a disaster. Enigma does NOT die quietly. If one does a “killall –9 enigma”, it crashes the box, corrupts the flash, and prevents it from working again.

So, if you want Enigma (or any other program in the Unix environment) to exit cleanly, why on earth do you use the 9 signal? Do you understand what it does, and what it means? IMHO, you get what you deserve.

[dilbengineer]

I therefore wrote some newmake targets flash-null-jffs2-1x and flash-null-jffs2-2x building jffs2-images for this purpose. (Creating cramfs- or squashfs-images for this purpose seems slightly silly.)

Would you mind explaining what the virtues are that make one approach less silly than another? I'm not doubting you, but it would help to understand why.

Would you mind sharing your alternate newmake with the forum? As you can read from my postings, I've tried to contribute one approach. Perhaps you can do one better.

So, if you want Enigma (or any other program in the Unix environment) to exit cleanly, why on earth do you use the 9 signal? Do you understand what it does, and what it means? IMHO, you get what you deserve.

I'm merely explaining what my experience was so that the readers of the forum know what not to do. I might have gotten what I deserve, but the forum readers can hopefully benefit from my mistakes.

Yes, I know the difference between SIGTERM and SIGKILL. Enigma seems to ignore the former; it doesn't have a choice with the latter.

[Barf]

I therefore wrote some newmake targets flash-null-jffs2-1x and flash-null-jffs2-2x building jffs2-images for this purpose. (Creating cramfs- or squashfs-images for this purpose seems slightly silly.)

Would you mind explaining what the virtues are that make one approach less silly than another? I'm not doubting you, but it would help to understand why.

Not at all. Using an "jffs-image" you have all your files in one writeable partition. Simple, logical, easy to change. Due to limited flash size, traditionally images for embedded devices have had things that do not change in a separate partition, using a compress (therefore read-only) filesystem (here cramfs or squashfs), while the writeable part (/var on the dBox) contained both files the system writes, as well as files the user is supposed to fumble with. There is a lot of extra work involved in setting up a "good" image with this scheme: In the case of a Unix-like operating system with root read-only, there has to be a number of links from /etc to /var/etc and so on. If making a mistake in the read-only part, the only option is to flash the partition again. For taking advantage of the scheme, the size of the read-only partition has to be as tight as possible -- unused space there is truly wasted. IMHO, you should not go through all this hassle unless you know that it is useful and needed (i.e. what you want on the device will not fit otherwise).

Would you mind sharing your alternate newmake with the forum?

Actually I have, in German. See the CDK forum, that contains a few hundered postings on the theme. Official documentation is: this (basic usage and concepts) and this (architecture) (in English!).

As you can read from my postings, I've tried to contribute one approach. Perhaps you can do one better.

Note that I do not claim to have solved your question...

Yes, I know the difference between SIGTERM and SIGKILL.

I am happy to hear that.

[dilbengineer]

Danke schön!

Thanks for the explanation and the links! Your official documentation is quite impressive.