flash drive "fusion"

[meowth08]

Hi,

Just an idea:

Is it possible to combine two 2-gigabyte flash drives so I would have a 4 gigabyte?
This would be helpful, say if I need to save a 3GB file.
I know of course that purchasing a flash drive would save me from creating this kind of problem .

However, this could lead to some good information.

 

[MrAl]

Hi,

Not that i know of, but you could use a small program to split the file into say 1.5GB and 1.5GB and store half on one USB drive and the other half on the other USB drive. To retrieve the file, you use the program to put it back together on your hard drive, then read it in the normal way. If you have two USB ports this is a breeze.

 

[DerStrom8]

Not physically, but there IS a way to combine the storage provided by two flash drives connected to a computer, and it's done through the computer itself. However, it's a very lengthy and difficult process and not recommended. There are a lot of things that can go seriously wrong, so I'd suggest just buying a $5 4GB flash drive

 

[MrAl]

Hi there 'Nick' (going by the picture ,

Well i've done this a million times so it's nothing new for me. I made my own program to pack and unpack the files. It's not real difficult and you might even be able to use DOS to do it with the copy command (i prefer not to do it that way however).

But of course using a larger USB drive is much easier, and they really have come down a lot in price now. You can get a 64 Gigger for a little over 30 dollars US if you watch for a sale.

What i do is put the files to be packed into a directory. Then run the program specifying the directory name. The program packs the files into one huge file.
Then i store the file on some medium like large USB. The program adds a header with file information such as checksum and file length.
To get the files back, i copy the huge file to hard drive, run the unpack program specifying a new directory, and the files appear and the program informs of any errors.

So the process he is looking for would be the reverse...
Run the program and specify the huge file and max size for one sectional part, then store the two files however he chooses.
To get the original file back, specify the location of the parts and run the program in the 'pack' mode. The huge file results.

It's not as simple as copy and paste, but it's not too difficult really especially after you do it once or twice.

One thing i always meant to do was to write a Windows based program to do this where the user can select the files via Explorer. For now though i just specify the directory or file name in a separate text file that the program reads as it starts up.

Another thing i meant to do was to add error correcting codes that would be included in the header. This way if there was an error (i never had an error yet) there is a good chance it could be corrected.

It compiles multiple files and restores them when needed with no compression, so it's just one step down from a zip file maker

 

[DerStrom8]

Hi there 'Nick' (going by the picture ,

Well i've done this a million times so it's nothing new for me. I made my own program to pack and unpack the files. It's not real difficult and you might even be able to use DOS to do it with the copy command (i prefer not to do it that way however).

But of course using a larger USB drive is much easier, and they really have come down a lot in price now. You can get a 64 Gigger for a little over 30 dollars US if you watch for a sale.

What i do is put the files to be packed into a directory. Then run the program specifying the directory name. The program packs the files into one huge file.
Then i store the file on some medium like large USB. The program adds a header with file information such as checksum and file length.
To get the files back, i copy the huge file to hard drive, run the unpack program specifying a new directory, and the files appear and the program informs of any errors.

So the process he is looking for would be the reverse...
Run the program and specify the huge file and max size for one sectional part, then store the two files however he chooses.
To get the original file back, specify the location of the parts and run the program in the 'pack' mode. The huge file results.

It's not as simple as copy and paste, but it's not too difficult really especially after you do it once or twice.

One thing i always meant to do was to write a Windows based program to do this where the user can select the files via Explorer. For now though i just specify the directory or file name in a separate text file that the program reads as it starts up.

Another thing i meant to do was to add error correcting codes that would be included in the header. This way if there was an error (i never had an error yet) there is a good chance it could be corrected.

It compiles multiple files and restores them when needed with no compression, so it's just one step down from a zip file maker

Thanks for the post, Mr Al! It's been very educational!

The process I've used in the past went a lot deeper than what you suggest, and it was lengthy, difficult, and risky. It's good to know there are better ways to do it!

Regards,
Matt ("Nick" works too, I suppose )

 

[RichTheDude]

Try arhive software (winzip 7zip and select a removable media size).

I have heard of HJsplit too.

 

[DerStrom8]

meowth08, what operating system are you using?

I stumbled across this article today that may prove to be very helpful: https://itexpertvoice.com/home/usb-flash-drive-raiders-of-windows-7/

Hope this helps!
Regards,
Matt

 

[meowth08]

Hi Matt,

This is a very interesting article.
Normally, I don't read a full article but this amazed me that I read through the last words.
I will definitely recommend this to friends.
Unfortunately, I think I could not use that since I'm using Win7 Home Basic.

Hi RTD,

I also read some an article comparing 7-zip, winzip, and winrar.
7-Zip can easily beat the other tools if the input files are uncompressed files like .doc, .bmp, .xls, etc.
I'm getting this.

Hi MrAl,

In your post you mentioned about adding error correcting codes.
We have tackled in school about error detection and correction.
And adding error correction codes is one form.
Others would be CRCs, checksum, parity bits etc.
Problem is they were not actually demonstrated.
So basically, I know that there is a thing called error detection and correction but I know nothing about how it is actually implemented.
Could you guide me on how you used the error correction codes or could you give a very basic demonstration? Thanks.

m8

 

[MrAl]

Hi Matt,

This is a very interesting article.
Normally, I don't read a full article but this amazed me that I read through the last words.
I will definitely recommend this to friends.
Unfortunately, I think I could not use that since I'm using Win7 Home Basic.

Hi RTD,

I also read some an article comparing 7-zip, winzip, and winrar.
7-Zip can easily beat the other tools if the input files are uncompressed files like .doc, .bmp, .xls, etc.
I'm getting this.

Hi MrAl,

In your post you mentioned about adding error correcting codes.
We have tackled in school about error detection and correction.
And adding error correction codes is one form.
Others would be CRCs, checksum, parity bits etc.
Problem is they were not actually demonstrated.
So basically, I know that there is a thing called error detection and correction but I know nothing about how it is actually implemented.
Could you guide me on how you used the error correction codes or could you give a very basic demonstration? Thanks.

m8

Hello again,


These codes send or store a little extra information so that we can tell if the file has been damaged, and the error correcting codes go one step further in providing that extra information that allows us to be able to determine what bits have been flipped that caused the error.

The checksum is the simplest to implement. In an 8 bit implementation we'd just keep adding all the bytes in the file one after the other and when the sum becomes greater than 255 we just subtract 256 from the result and that is the new checksum. We then store the checksum with the file somehow and later check it when we go to read it again.

Parity isnt too hard either, just count the bits that are 1's and then add another bit that is 1 if the count is odd and 0 if the count is even (or vice versa).

CRC is a little harder to implement as instead of simply summing all of the bytes we sort of look up the value to add in a special table. The table gives us the byte to add rather than use the byte from the file directly. This provides more protection than a simple checksum because the value of the sum can change wildly and that means even a small change like one bit in one byte can force the sum to swing from a large value to a low value or vice versa. CRC32 is considered pretty good but todays file codes go even much higher than that. The main idea is to reduce the probability that a file will still have the same CRC code even with one small change to the file.

Hamming codes go one step farther than the CRC coding. These codes introduce regular data into the file that can be used to tell what bits if any have been flipped. Of course this takes more information to be stored with the file. To see how these codes work you should probably look this up in Wikipedia because it's much more involved. The power of the codes is still limited to a certain number of bit flips and more error than the predetermined number will still not allow correcting of the file bytes, but there's a better chance that it will work. With some file errors i have seen though this might not work because sometimes whole sections of the file get corrupted. So it depends how bad the file is.

 

[DerStrom8]

Hi Matt,

This is a very interesting article.
Normally, I don't read a full article but this amazed me that I read through the last words.
I will definitely recommend this to friends.
Unfortunately, I think I could not use that since I'm using Win7 Home Basic.

I'm glad you found it interesting. I can't imagine it would be any different for Win7 Home Basic. It seems to me that it should work just fine on that, too. Have you tried it yet?

Regards,
Matt

 

[WTP Pepper]

Hi Matt,

We have tackled in school about error detection and correction.

m8

Good Lord. Where did you go to school? I learnt this on my 3rd year in a Telecoms degree. At school we never got past simultaneous equations.

 

[MrAl]

Hello again,

Here's a quick example of calculating the checksum. I'll do an 8 bit checksum with a very short file (or message) to keep this simple...

Lets say the bytes of the file are (all in hex):
0x0C, 0x26, 0x91, 0xE7, 0x34

Starting with the first byte on the left, 0x0C, we add the 0x26 and we get:
0x32
Next we add the 0x91 and we get:
0xC3
then we add the next byte, the 0xE7 and we get:
0x1AA
and as you see we got a carry into the third digit which would take more than 8 bits, so we subtract 0x100 (which is 256 decimal) from that now before we add anything else and we get:
0xAA
which again only takes 8 bits.
Now we add the last byte in the file which is 0x34 and we get:
0xDE
and we are done, so 0xDE is the 'checksum' for this file.
Now as the file is stored this checksum is also stored either with the file itself (could be at the very end of the file for example) or in a table somewhere. Then later when the file is to be read again the checksum is again computed as each byte is read and then compared with the stored checksum. If there is a difference between the checksums then we know something is wrong with the file.

The checksum is better than nothing, but it's not considered a strong error detection scheme. That's why CRC came into existence. It's a bit more complicated but not too hard to implement really.

 

[meowth08]

Hi everyone,

Death of grandfather and consequently, missed requirements kept me busy for the last few days.

Matt,

"Also keep in mind that only Windows 7 Professional, Enterprise, and Ultimate support software RAID. "
This is part of the article you posted so I thought it would not work on homebasic. I have not tried it though. I'll have to find a flash drive donor friend.

WTP Pepper,

Secret

MrAl,

Thanks for that short discussion together with an example on checksum. I have checked for parity, CRC, and hamming code in the net and looking at the examples, I could still remember them so well. But what I am looking for is a real application like what you mentioned here.

"Another thing i meant to do was to add error correcting codes that would be included in the header. This way if there was an error (i never had an error yet) there is a good chance it could be corrected."

How did you add an error correcting code?

Regards,
meowth08

 

[4pyros]

I can't imagine it would be any different for Win7 Home Basic.

It said RAID is not supported in the basic versions.
Also some of the comments say win 7 can not strip a USB drive so it will not work at all.

 

[MrAl]

Hi everyone,

Death of grandfather and consequently, missed requirements kept me busy for the last few days.

Matt,

"Also keep in mind that only Windows 7 Professional, Enterprise, and Ultimate support software RAID. "
This is part of the article you posted so I thought it would not work on homebasic. I have not tried it though. I'll have to find a flash drive donor friend.

WTP Pepper,

Secret

MrAl,

Thanks for that short discussion together with an example on checksum. I have checked for parity, CRC, and hamming code in the net and looking at the examples, I could still remember them so well. But what I am looking for is a real application like what you mentioned here.

"Another thing i meant to do was to add error correcting codes that would be included in the header. This way if there was an error (i never had an error yet) there is a good chance it could be corrected."

How did you add an error correcting code?

Regards,
meowth08

I dont understand your question. I thought i said that i "meant" to add error correcting code. But i never actually added it yet. I dont know if i will or not.
What else i dont understand is if you looked up Hamming Codes, then how could you not see how to add codes? There is some method you follow and that gives you your data to be included with the file or set of files. You should realize however this isnt going to be cheap bytewise, meaning the stored file size will increase. Without the codes the file size only increases slightly, in fact very slightly.

You might start by thinking about what it takes to store files when you dont already have a file system. For one thing a table is nice, that tells you where each file is located within the whole batch. You can store CRC data there too. Another way is simply to use a small header for each file, where before the file is stored you first store the file name, date, and CRC code, and number of bytes [this is the header]. So this is what it would look like it a huge file:
[12456,"MyFile1.jpg",0x564A3EE9,122411,012345]followed by the actual file bytes for MyFile1.jpg, then
[345677,"MyFile2.jpg",0xAE84F9E8,122111,022534]followed by the actual file bytes for MyFile2.jpg, then
the next HEADER and FILE,
the next HEADER and FILE after that,
etc.
In the above first file, 12456 was the byte count for the file, only the file itself, then the name of the file, then the CRC32 code, then the date, then the time, then finally the actual bytes of the file.

There are really a lot of ways to do this, you might want to come up with your own method. You could also add encryption if you like.

 

[meowth08]

Hi MrAl,

The second paragraph example answered the question I had in mind.
I am sorry I wasn't able to express my questions clearly.
I am learning to express my questions clearly bit by bit and eventually,
after more experience, I can do it nibble by nibble, then byte by byte,
then word by word, then Dword by Dword then Qword by Qword...

How do we add encryption?

 

[MrAl]

Hi,

Geeze, there are millions of encryption methods out there i can give you a quick idea about one but you'd probably want to look up more on the web.

For the simplest example, you choose a phrase like "The red pear looks green on sunny days", or something even less logical, and maybe lose the spaces: "Theredpearlooksgreenonsunnydays" and then you start by XORing each letter in that phrase with each letter in your message (file), and that becomes the new message. For example, say your real text is "Account Number 12345". You start by doing an XOR of the ascii T with the ascii A, and the result becomes the first byte in your file. Next you do the next letter of the phrase and the next letter of the file, which are 'h' and 'c', XOR them, and that becomes the second byte of your encoded file. You continue this process and when you get to the end of the phrase and you still have more text to encode, you start over at the beginning of the phrase using it again and again throughout the entire message. That generates the bytes for the file.
To recover the original text, you do the exact same thing again, only this time after you XOR using the same message, you get the original file bytes back. So if you start with "Account Number 12345" it will turn into garbage once XOR'd with the message, but when you XOR the garbage the same way you did the message, you get the original message back again.
That's one of the simplest schemes. You just have to remember what your encoding phrase was or else you cant get the message back too easy.

If you are interested in encoding and want to read something pretty interesting about encoded messages from the past, check out the Voynich Manuscript. It's an old book with text that no one has yet been able to decode with any certainty, and are not even sure what alphabet it is using or even if it is a true alphabet or a clever fake. It's more of an interesting read however more than an educational tool.

 

[meowth08]

Hi again sir,

Your example is great.
I heard of the word encryption during the recent Philippine automated election.
They said that hacking the results of just one precinct, because of its 128 bit encryption system, will take 50 years to decode.
Now, I get it.