Modify Linux commands to use Kingston Flash Media reader.

Clarifications

Is BIOS trying to boot from a removable device first, where the reader is somehow indicating that it is capable of booting the system, but isn't?

Once formatted, do you need to wipe the format with sfdisk, or dd to sdb1 rather than sdb?

Perhaps if you step through the procedure at the command line, rather than through the script, you may get more feedback on what is happening, as scripted commands seem to abort silently. You could also check the partitions with sfdisk (man sfdisk for your options)to see what's up.

I'm just shooting in the dark here, I'm no unix guru, as evidenced by the fact that I'm completely baffled by the procedure: copy file > partition > format > install.

It's starting to sound like one of those Windows/USB ports things.

Their internal search engine is down:

http://search.debian.org/

I've tried searching the web for various combinations of usb, media, flash, reader, player, debian, /dev, /sda, /sdb, etc. Most of what I get is sites trying to sell USB devices. I also get people trying to download music to or from thumb drives. I haven't found anybody trying to do anything like what I'm attempting, but maybe I'm not using the right search terms. I've got the original problem description in a text file, but I've run out of ideas about where else to post it.

I have confirmed the person who developed the original procedure has left the company. The guy I've been working with is glacially slow to respond to requests but makes up for it by appearing to know less about Linux than I do. He definitely doesn't know why I'm dd'ing a .img file to the flash media and then apparently destroying it by partitioning.

"I'm not sure why they documented each command with 'echo' lines"

I'm guessing they copied an existing bat file or script to the word document. If run as an executable, the given list would anounce what it is doing then perform the action; potentially with little visible output returned.

This is an interesting puzzle you've got though. I'll be watching how it goes and will give the post a more detailed read later when time permits. I'm not sure that I can add to what other's have probably suggested already but we'll see.

I'm doing all this from the CLI.

I'd prefer to leave the media player connected permanently. This is a production system for the factory floor. It will be easiest on those using the system if they don't have to keep straight the boot / connect sequence. Another reason is the box I'm using has it's USB ports only on the back.

I'm not sure what you mean by "Look in your /dev and you should spot a /dev/by-id, /dev/by-label ..." I did a 'cd /dev' followed by 'ls b*' but apparently wild cards don't work the way I expect; the only result was a single line with 'usb' on it.

So you basically need a media reader slot and a button that runs the scripts to do the prep work. I think I'm all caught up now and left some notes in the previous comment.

Consistant location mounting of USB media is one of the things I'd like to see done more cleanly. Nothing as drastic as mounting to letters, just more consistant. It's probably more my lack of learning about it though.

Anyhow, confirm that you have the /dev/disk/* directories and see if there is a common way to identify the new media device as it's picked up. Hopefully, one of those directories is showing the non-changing reader identifier.

In *nix, everything is a director and file. /proc is actually a non-existant directory tree generated out of the kernel. Your litterally going inside the running kernel and poking around when you do a "ls /proc" or more "/proc/cpustatus". In like fashion, /dev/disk/*/usb-Kingston* is a file representing that device through a link back to the actual device reprsented by /dev/sd*.

Ignore that bit if it's just repeat information.

The plan is to find a consistant identifier for the item on the dynamicaly assigned USB bus so my thinking was that you may find something in that /dev/disk tree.

"what do the fresh media present if you stuff one of those in the reader"

cd /dev/disk

remove the media reader

ls -l ./by*

see what identifiers are there (the usb-King* thing)

Attach the reader without media in it and repeat. See whatnew identifiers apear (my understanding is that HAL should do so).

Insert the media and repeat again. See if new identifiers apear for the media in the reader.

Between the reader and the media, see if there is a consistant identifier or how much consistancy you can find between two different "disks" inserted into the reader.

I'm not yet sure how to modify your script to have it use a "mostly" consistant identifyer for the disk yet but this get's you a step closer.

I would then do some mounting of those by* identifiers with a formatted bit of media to confirm which identifers map to where you

md /mnt/test
mount /dev/disk/by*/usb-King* /mnt/test
ls /mnt/test
(are these the droids you are looking for?)

umount /mnt/test
mount /dev/disk/by*/usb-King*[otherone] /mnt/test
ls /mnt/test
(are these the droids you are looking for?)

If one of those Kingston identifiers is consistantly gets you the media partition mounted; you have something you can script against.

If the inserted media add more identifiers then you need to confirm what degree of consistancy you have between them based on what mounts your preformatted one properly. You then have something to script against but not as easily as the first case.

(My USB media tends to be repeat use rather than one time write on ongoing fresh media but I do like the puzzle. I suspect it'll be one of those "well that was bloody easy" things in the end too but like any learning; it's a pig to do the first time.)

Another thought that won't work but I gotta mention. If it's a factory floor; there may be benefit in considering a minimal X interface. Insert media, pointy-clicky the [go] button. remove the media whent it says "done and unmounted".

(I'm sure other's know a more clean way to get through this the first time but I'm more a "hack away at the keyboard until it does what you want" type with any system.)

So, hypothetically, you get the script mounting consistantly and running clean.

You could even drop each command into it's own chmod 700 file so you can simply modify the command line until each step works right on it's own then:

(if cmd01, cmd02, cmd03 as names)
cat cmd* > script01

To compile the single command scripts into your final script.

But, the "more automation" idea. Once the script runs clean and consistantly. You could even add a cron job that checks for the disk identifier every few seconds.

a. If it exists, run the script.

b. If it exists and the user pressed "continue" (spacebar, Y.. whatever), run script.

c. If it exists and is not already written, rn script.

Option A is easy to setup but will wipe and rewrite your media every few seconds. That may eat the write limit of the flash chips while your worker is at lunch.

Option B gives you the user confrimation that this is a new disk and should be written.

Option C goes more the automation route with various ways to confirm that a new media has been inserted. You could even automatically start the process then have a confirmation at the end saying "please remove media and press any key". That would keep the scripts PID active which gives you somethign to run the scripts if statement off of. (if PID exists, end else, step through commands)

I'm omitting the 'ata-MAXTOR' results that I assume apply to the HD. After removing the media reader:

cd /dev/disk/by-id
ls

No usb-Kingston entries. Reattached the media reader with no flash card:

ls
usb-Kingston_FRC-HS219.1_000003537

Inserted a flash card in the reader:

ls

No new entries; specifically, no 'usb-Kingston_FRC-HS219.1_000003537-part1'. If I had to guess I'd say it isn't seeing the flash card. Unfortunately I only have one flash card to play with.

I'll get back with you on the 'mount' tests.

You lost me at 'chmod 700 file'. From there it was down an icy hill on a greased banana peel

I wish my knowledge would catch up to my imagination
most days.

Short answer; chmod changes permissions and 700 makes
the file executable, readable and writable for the file
owner only. If the file has executable permissions than it
can do stuff else it's just a text file.

ok. briefly, chmod is your command to change the
security settings on a file or directory. Change Mode is the
long name I believe. You also have chown and chgrp.

(Linux in a Nutshell would be a good book to browse
through at your chapters too; great command reference)

So, with chmod you change a files user
read/write/execute, or group, or anyone (reading left to
write). You can do this two ways; by letter or by number.

chmod u+x u+w u+r filename
chmod 700 filename

In the first case, you specify granting the user execute
permissions turing a text script into an executable script.
You write the .bat file then you have to allow the .bat file
to actually run commands in Windows lingo.

In the second case, you are doing the same but with a
numeric value. The file permissions you'll see with ls -l
are rwxrwxrwx. The first three are User, the second are
Group the third three at furthest right are anybody on the
system. This can also be indicated numerically as
421421421 a-la age old binary counting:

700 = user read/write/exec, no group permissions, no
any permissions = rwx------. if permissions exist for
group or user, they are removed. For a folder, the owner
can enter it, read files within it or write files within it. For
a file, the owner can execute it, read or write it.

777 = rwxrwxrwx. Everybody can do what they like with it.
it's just a bad idea in general unless it's trully an
"everybody" area you chmod'ing.

600 = rw-------. owner can read/write but no one else.

644 = rw-r--r--. Owner can read/write, all others can
only read.

man chmod will give you all the gory details.

chown and chgrp are two other commands you'll want to
have basic familiarity with. Change owner and change
group. I tend to use chown more than chgrp simply
because I can change bot user and group with it.

chown username:groupname filename

In a user's home directory, it can be handy to do a:

chown -r username:group ./*

That recursively insures that the user owns everything
under there home directory. I frequently move crap to my
home directory using the root command so giving my
user account ownership is required before I scp whatever
it is back to my local machine or send something the
other direction.

As for my wild ideas:

- the sript alone can be started manually in it's current
form leaving the intelligence too the operator.

- the script could take on some of the intelligence by
confirming that a disk is there and asking the user if it
should proceed. This would be using the script as a cron
job with a "press any key to continue" pause.

- the script could be mostly automated with intelligence
in the third case where it confirms if it is still working on
a disk or if the disk is new and needing to be imaged still.
it then takes on all intelligence except telling the operator
that the disk is ready. That's probably where you want to
be aiming for but the latter two ideas take me a bit of
learning to go from concept to actual code.

I've had a couple of other items take precedence. I'll return to this project next week.

Edited - and apparently I'm better than I thought since both of them are out of the way. See 'Starting Over' below for the lastest! I'm in danger of making actual progress!

I reloaded Debian this afternoon. This time I took the 'Desktop' option and wound up with the Gnome GUI. I also set up the network connectivity and our proxy server, so now I don't have to manually enter results on my Windows box to post to the Internet.

It looks like the media reader is consistently appearing on /dev/sda. At this point I've decided to assume it will continue to do so and move on. Here's what I've got so far, entered at a command prompt as root:

# cd /home
# dd if=nia-720.img of=/dev/sda
# cd niafs720
# sfdisk -f /dev/sda /dev/null
Checking that no-one is using this disk right now ...
BLKRRPART: Device or resource busy
This disk is currently in use - repartitioning is probably a bad idea.
Umount all file systems, and swapoff all swap partitions on this disk.
.
.

And some other stuff. So

# mount -l
/a bunch of other apparantly unrelated entries
/dev/sda1 on /media/usb0 type ext3 (rw,noexec,nosuid,nodev)

I didn't mount this manually so something else must have.

# umount /media/usb0
# sfdisk -f /dev/sda /dev/null
# sfdisk -f /dev/sda <128mb1.part> /dev/null
Checking that no-one is using this disk right now ...
OK
Warning: The partition table looks like it was made
for C/H/S=*/3/28 (instead of 1016/4/62).
For this listing I'll assume that geometry.
start: (c,h,s) expected (0,1,5) found (0,0,33)
end: (c,h,s) expected (1023,2,28) found (1011,2,28)
Warning: partition 1 does not end at a cylinder boundary
If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes: dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)

So at this point I've got a bunch of new questions.
1) Should I assume I need to 'umount /media/usb0' every time and stick a line in my procedure

2) My understanding of sfdisk is it's a partition management tool, but the man page doesn't do a good job of explaining the parameters. Are the two commands setting up two separate partitions, or is the second one overwriting what the first one did? What roles do '', '<128mb1.part>', and '/dev/null' play? There are two files in the niafs720 directory with the same names as the .part parameters, so I assume the sfdisk commands are doing something with them, but I can't figure out what. Also, I thought /dev/null was a null device, but I can't tell what role it's playing here.

3) If the sfdisk is partitioning the flash media, are the results of the initial 'dd' command being lost?

4) While we found entries in /dev/disk/by-id and /by-uuid that appear to correspond to the media reader, I'm unclear what I'm supposed to do with them.

5) Are the warnings displayed after the second 'sfdisk' command anything I need to be concerned about?

That ought to be enough to let me keep Google's servers thrashing this weekend.

What determines if a file is executable is the permissions one has to it, not the content of the file itself? What happens if I make something non-executable, like a spreadsheet, executable and then try to 'run' it?

Correct. "executable" is a permission attribute specified explicitly for the owner or group or anybody or combination of those three.

If you make a spreadsheet executable, it will return an error or at least I assume as much. I've not thought to try that but I'm going now that you bring it to mind. I expect either an error or a coredump file as the results but we'll see.

In shell scripts, you may notice many of them starting with a shell entry:

#!/bin/bash (or often #!/bin/sh)
## file made to do something for someone
## date file was made

command
command

That first #! line will tell the system what shell it should prefer to use though if the line is not present and the script is executable, it will run in the current shell.

Now, here's the advantage; when something is downloaded, it does not get executable permissions. It will likely have owner read/write permissions and equal or lesser group and other permissions. The user then has to make the file executable if that is the valid and intended use. If someone else is able to sneak something onto your system through the network, they still gotta crack an account to make the file executable; the file probably being the tool expected to crack the account in the first place.

This can be broken by adjusting the default permissions mask for new files but that's like turning off the Windows firewall then surfing for porn and cracks.

permissions and chmod/chown/chgrp are one of the corner stones to be aware of unless a user is working purely through the GUI on a preconfigured appliance system.

That is purely about "permissions". At the risk of making things more complicated, there are also true file "attributes". With fat/ntfs, you have R, H, S, A that can be applied to files or directories. In the same way, "chattr" is a command to change attributes of directories/files with about seven or so attributes to choose from. I'm a fan of the "only allow changes from owner/creater" attribute for things like website directory trees.

Most of the time, permissions are all that one will be mucking with making things readable, executable or restricted to a smaller control group. I just thought I'd mention attributes in addition for interest. I don't think they apply to this task but they are good to keep in your mind distantly encase they become usefull in another case.

I've partitioned drives and moved crap around on platters all over the place. Given the starting materials and end objective, we may be able to write Palmetto a clean script to do the same thing.

Simply copying the image and making it bootable is how I would approach it. Not having any way to verify that the result is the desired product, fully functional, is entirely insane. It seems there was a lot of ball-dropping going on before this little gem of a project was handed to Palmetto.

There is way too much uncertainty, and far too little documentation, to be playing around with the safety factor. Where are the requirements, etc., that started this project prior to the disappearing guy even getting his hands on it? Oy vey.

edit: If the image is not ext3fs, I could see that it may need to be converted, but the sfdisk would wipe everything, no?

The other step that needs to be taken is to do the installation (unpacking) from the niafs720 directory, and the subsequent deletion of the /var/ directory (for some reason). Perhaps because that /var/ belonged to the machine the image was created on? (Useless for any other machine.)

What I'm wondering is, if a successful installation does occur, could Palmetto then just create an image of the installation, then just copy (dd) it to any number or drives, no muss, no fuss?

My poor choice of words has mislead you. I personally don't have a way of testing the media or even know what it's supposed to do. However, others in this building have a test procedure for the assembled component that uses the media. The problem is we're not scheduled to make any of the assemblies for several weeks yet. This is good because it gives me plenty of time to get it done, but it makes testing difficult. Hopefully when I've got something ready to test we can pull a finished unit off the shelf.

Sorry for the lack of feedback on this lately; I've had other support issues push it to the back burner.

Any time I have a flashdrive currupted, I do a "dd if=/dev/sda of=~/flashdrive.iso" just so I can later mount the ISO if needed (yeah, mount isn't just for devices.. muwahaha).

In the same way, unless each disk is different somehow, he can simply dump a working flashdrive image to an ISO with DD.

He should even be able to mount the .img file directly and see what's actually in there.

I figured you'd be a bit spotty when you mentioned that other projects where getting priority at the moment. Good to hear your not working blind either.

Is there any of the older units availabe from the testing team? Seeing a final working flashdrive and what it does may clarify your requirnments.

I wish we could get some of the other even more knowedgable eunics in here. (I couldn't resist. ;D)

(Like anything, this wouldn't be half as complicated if it wasn't through text back and forth; it's like negotiating a contract over email.)

http://maemo.org/community/wiki/HowTo_EASILY_Partition_your_MMC_card/

I've finally cut that page into a PDF but that is my reference for sfdisk when I have to repartion a free SD for my PDA.

You may be able to step through the interactive sfdisk process to get the process working then translate that to the non-interactive command lines.

I now have to go do a mkdosfs and mkextfs to create the file system within the two partitions I cut but I thought to hand off the link now before I forgot.