PDA

View Full Version : Can anybody "Undelete" records in a Native DF File?



Michael Mullan
29-Mar-2005, 05:02 PM
HELP!

I ran a purge routine on a (large) table in DF31 on AIX. Unfortunately,
about a week after I did this, I discovered that the 'archive' files had
been corrupted by a bad FTP transfer.

SO, No backup ( they only keep 3 days worth) and 900,000 records to
recover. Does anybody have the knowledge to 'reset' the "record is
deleted" flag in the standard database?

OR

Does anybody have the knowledge of what transferring a binary file as
ASCII actually does to the file, and if it is reversible?

Michael.

Sture Andersen
29-Mar-2005, 05:41 PM
Michael,

I am afraid, that there is nothing that will reverse an ASCII transfer of a
..dat file. Numeric and Date fields will have been damaged.

As for undeleting, I don't know.

-Sture

Michael Mullan
29-Mar-2005, 07:14 PM
Sture Andersen wrote:
> Michael,
>
> I am afraid, that there is nothing that will reverse an ASCII transfer of a
> .dat file. Numeric and Date fields will have been damaged.
>
> As for undeleting, I don't know.
>
> -Sture
>
>
Damaged how?

Is the process definable, and therefore reversable, or is it defined as
being one way only?

Michael.

Jan-E
29-Mar-2005, 07:32 PM
Michael Mullan in dataflex (Tue, 29 Mar 2005 17:02:40 -0500):

>I ran a purge routine on a (large) table in DF31 on AIX. Unfortunately,
>about a week after I did this, I discovered that the 'archive' files had
>been corrupted by a bad FTP transfer.
>
>SO, No backup ( they only keep 3 days worth) and 900,000 records to
>recover. Does anybody have the knowledge to 'reset' the "record is
>deleted" flag in the standard database?

I just tested what happens in 2.3b when you delete a record: all the data
get wiped out. I suppose it is the same with DF31. No chance of recovery.

>Does anybody have the knowledge of what transferring a binary file as
>ASCII actually does to the file, and if it is reversible?

Depends on which way it goes. From X to Dos: replace every CR with CR/LF.
From DOS to X: replace every CR/LF with a single CR. Did the file shrink
or expand after the bad FTP transfer? Reversible? Depends on the data in
the file. Short records with a lot of strings give more hope of success.

Jan
--
Low CPU versions of DF2.3b - http://www.monitor.nl/df/df.html

Jan-E
29-Mar-2005, 07:47 PM
Jan Ehrhardt in dataflex (Wed, 30 Mar 2005 02:32:29 +0200):

>Depends on which way it goes. From X to Dos: replace every CR with CR/LF.

To be more precise: replace every single CR with CR/LF. An already
existing CR/LF remains untouched. You must make an intelligent guess to
reverse the process.

Jan
--
Low CPU versions of DF2.3b - http://www.monitor.nl/df/df.html

Jan-E
30-Mar-2005, 04:26 AM
Jan Ehrhardt in dataflex (Wed, 30 Mar 2005 02:47:20 +0200):

>Jan Ehrhardt in dataflex (Wed, 30 Mar 2005 02:32:29 +0200):
>
>>Depends on which way it goes. From X to Dos: replace every CR with CR/LF.
>
>To be more precise: replace every single CR with CR/LF. An already
>existing CR/LF remains untouched. You must make an intelligent guess to
>reverse the process.

Michael,

Could you try to do the reverse: upload as ASCII if the first bad transfer
was a download or vice versa? And then report the size difference before
and after the second transfer. The size difference tells you how many
intelligent guesses you have to make to recover the file. If it is low
enough (but what is low?) there might be a chance to recover the complete
file.

Jan
--
Low CPU versions of DF2.3b - http://www.monitor.nl/df/df.html

Greg Sergeant
31-Mar-2005, 12:32 PM
I just double-checked what Jan said about the data being wiped when a
record is deleted. He is correct. Not that I doubted Jan, I was just
checking out of my own curiosity.

I created an application in Visual Basic that opened a dat file in
binary mode and read it a block at a time. After some fiddling I got it
to skip the header and begin reading at record number one and looping
through the first 25 records or so. Then I deleted the first record and
reran. After the delete, the data for the first record was completely
wiped out.

My guess would be there is no chance of recovery IF the records were
deleted by Dataflex. Hopefully you will have some luck reversing the
transfer process.

Greg


Jan Ehrhardt wrote:

> Michael Mullan in dataflex (Tue, 29 Mar 2005 17:02:40 -0500):
>
>
>>I ran a purge routine on a (large) table in DF31 on AIX. Unfortunately,
>>about a week after I did this, I discovered that the 'archive' files had
>>been corrupted by a bad FTP transfer.
>>
>>SO, No backup ( they only keep 3 days worth) and 900,000 records to
>>recover. Does anybody have the knowledge to 'reset' the "record is
>>deleted" flag in the standard database?
>
>
> I just tested what happens in 2.3b when you delete a record: all the data
> get wiped out. I suppose it is the same with DF31. No chance of recovery.
>
>
>>Does anybody have the knowledge of what transferring a binary file as
>>ASCII actually does to the file, and if it is reversible?
>
>
> Depends on which way it goes. From X to Dos: replace every CR with CR/LF.
> From DOS to X: replace every CR/LF with a single CR. Did the file shrink
> or expand after the bad FTP transfer? Reversible? Depends on the data in
> the file. Short records with a lot of strings give more hope of success.
>
> Jan

Jan-E
31-Mar-2005, 12:48 PM
Greg Sergeant in visual-dataflex (Thu, 31 Mar 2005 11:32:09 -0600):

>He is correct. Not that I doubted Jan ...

<grin>

>I created an application in Visual Basic that opened a dat file in
>binary mode and read it a block at a time. After some fiddling I got it
>to skip the header and begin reading at record number one and looping
>through the first 25 records or so.

I am always surprised to see what people have to do to view the contents
of a file. I am using some good ol' DOS-utilities for that. Point, shoot,
view. Even the ILovYou virus I analyzed this way some hours before it
became world-wide news ;-)

Strange to notice that Michael has never returned here since his first 2
messages. He's probably hard working at making a couple af thousand
intelligent guesses...

Jan
--
Low CPU versions of DF2.3b - http://www.monitor.nl/df/df.html

Michael Mullan
4-Apr-2005, 12:28 PM
Actually jan I've been crying in my beer!.

The client has nothing resembling a useful backup, the DF records are
toast, and I decided that the FTP transfer sent the binary file as 7-bit
ascii, and so is totally non-recoverable.

I called the client, and he took it surprisingly well actually. He
hasn't sent any large gentlemen over to my house with concrete shoes for
me yet.. :-)

Thanks for the help anyway.

Lesson learned:

Tell them that you need a 'confirmed' backup before running any process
with the word 'delete' in it.


Michael.


Jan Ehrhardt wrote:
> Greg Sergeant in visual-dataflex (Thu, 31 Mar 2005 11:32:09 -0600):
>
>
>>He is correct. Not that I doubted Jan ...
>
>
> <grin>
>
>>I created an application in Visual Basic that opened a dat file in
>>binary mode and read it a block at a time. After some fiddling I got it
>>to skip the header and begin reading at record number one and looping
>>through the first 25 records or so.
>
>
> I am always surprised to see what people have to do to view the contents
> of a file. I am using some good ol' DOS-utilities for that. Point, shoot,
> view. Even the ILovYou virus I analyzed this way some hours before it
> became world-wide news ;-)
>
> Strange to notice that Michael has never returned here since his first 2
> messages. He's probably hard working at making a couple af thousand
> intelligent guesses...
>
> Jan

Bob Worsley
4-Apr-2005, 03:57 PM
"hard working at making a couple af thousand intelligent guesses..."

Nahhhh, Michael is generally better at doing SWAG <vbgd&r>
Bob

"Jan Ehrhardt" <Jan.Ehrhardt@monitor.nl> wrote in message
news:cgdo415s2mb3u3ln2ro9khn165mdsvii7h@4ax.com...
> Greg Sergeant in visual-dataflex (Thu, 31 Mar 2005 11:32:09 -0600):
>
> >He is correct. Not that I doubted Jan ...
>
> <grin>
>
> >I created an application in Visual Basic that opened a dat file in
> >binary mode and read it a block at a time. After some fiddling I got it
> >to skip the header and begin reading at record number one and looping
> >through the first 25 records or so.
>
> I am always surprised to see what people have to do to view the contents
> of a file. I am using some good ol' DOS-utilities for that. Point, shoot,
> view. Even the ILovYou virus I analyzed this way some hours before it
> became world-wide news ;-)
>
> Strange to notice that Michael has never returned here since his first 2
> messages. He's probably hard working at making a couple af thousand
> intelligent guesses...
>
> Jan
> --
> Low CPU versions of DF2.3b - http://www.monitor.nl/df/df.html

Anders Öhrt
5-Apr-2005, 03:23 AM
> Tell them that you need a 'confirmed' backup before running any process
> with the word 'delete' in it.

Whenever we do any kind of data-touching process (editing tables, running
deletes, fixes, etc), we always take a manual copy first (via xcopy
normally, or just via explorer copy/paste).

Of course you must always have a confirmed backup, but this adds an extra
layer of safety.

// Anders

Anders Öhrt
5-Apr-2005, 03:32 AM
I just remembered a program called "uncook", that is made to fix binary
files that are transfered as ascii. It works mainly on mp3's but according
to the FAQ it might work on any file. Check out
http://www.free-music.com/uncook95.htm, it might work.

// Anders

Michael Mullan
5-Apr-2005, 07:33 AM
Anders Öhrt wrote:
>>Tell them that you need a 'confirmed' backup before running any process
>>with the word 'delete' in it.
>
>
> Whenever we do any kind of data-touching process (editing tables, running
> deletes, fixes, etc), we always take a manual copy first (via xcopy
> normally, or just via explorer copy/paste).
>
> Of course you must always have a confirmed backup, but this adds an extra
> layer of safety.
>
> // Anders
>
>
I usually do too, but this DB fills up > 50% of the filesystem on the
server.

Michael Mullan
5-Apr-2005, 07:35 AM
Anders Öhrt wrote:
> I just remembered a program called "uncook", that is made to fix binary
> files that are transfered as ascii. It works mainly on mp3's but according
> to the FAQ it might work on any file. Check out
> http://www.free-music.com/uncook95.htm, it might work.
>
> // Anders
>
>

Ladies and Gentlemen, We have a WINNER!

Uncook95.exe does actually work.

Yippee!

Michael.

Michael Mullan
5-Apr-2005, 07:35 AM
Funny Bob!



Bob Worsley wrote:
> "hard working at making a couple af thousand intelligent guesses..."
>
> Nahhhh, Michael is generally better at doing SWAG <vbgd&r>
> Bob
>
> "Jan Ehrhardt" <Jan.Ehrhardt@monitor.nl> wrote in message
> news:cgdo415s2mb3u3ln2ro9khn165mdsvii7h@4ax.com...
>
>>Greg Sergeant in visual-dataflex (Thu, 31 Mar 2005 11:32:09 -0600):
>>
>>
>>>He is correct. Not that I doubted Jan ...
>>
>><grin>
>>
>>>I created an application in Visual Basic that opened a dat file in
>>>binary mode and read it a block at a time. After some fiddling I got it
>>>to skip the header and begin reading at record number one and looping
>>>through the first 25 records or so.
>>
>>I am always surprised to see what people have to do to view the contents
>>of a file. I am using some good ol' DOS-utilities for that. Point, shoot,
>>view. Even the ILovYou virus I analyzed this way some hours before it
>>became world-wide news ;-)
>>
>>Strange to notice that Michael has never returned here since his first 2
>>messages. He's probably hard working at making a couple af thousand
>>intelligent guesses...
>>
>>Jan
>>--
>>Low CPU versions of DF2.3b - http://www.monitor.nl/df/df.html
>
>
>