PDA

View Full Version : df3.1d memory



Glenn Hickman
10-Jun-2005, 05:46 AM
Hi,

We are having a problem with a program, it runs fine for a while but then we
get 'OUT OF MEMORY STATUS 10' error what causes this? and is there a command
to clear out memory we could maybe run whenever this program is run? The
program uses 15 arrays is this the problem?

Best Regards,
Glenn Hickman

Glyn astill
10-Jun-2005, 07:04 AM
And does anybody know a good way to get available memory so that we can
display it in the program? The memory command is useless.

Glenn Hickman wrote:
> Hi,
>
> We are having a problem with a program, it runs fine for a while but then we
> get 'OUT OF MEMORY STATUS 10' error what causes this? and is there a command
> to clear out memory we could maybe run whenever this program is run? The
> program uses 15 arrays is this the problem?
>
> Best Regards,
> Glenn Hickman
>
>

Roger Loftus
10-Jun-2005, 10:23 AM
Glenn Hickman wrote:

> Hi,
>
> We are having a problem with a program, it runs fine for a while but
> then we get 'OUT OF MEMORY STATUS 10' error what causes this? and is
> there a command to clear out memory we could maybe run whenever this
> program is run? The program uses 15 arrays is this the problem?
>
> Best Regards,
> Glenn Hickman

Glenn:

Wow - 15 data arrays? If these arrays accumulate a significent number
of items, I would surely think that this is the source of your status
10 at runtime.

You probably need to exercise control over these arrays if possible.
If it is an option, sending delete_data to an array not currently
needed frees the memory consumed by its contents. I believe also using
delete_item on array elemements frees memory in a similar manner. You
may need to experiment with writing an array out to a temporary data
file and reading it back in to perserve needed array data but reduce
concurrent array item counts.

The only other option is probably to focus on maximizing the free
memory of your session, and that is dependent on your OS and whether
you are using DOS or console mode.

Roger Loftus

Ashley Stead
10-Jun-2005, 11:08 PM
Hi,
I also have a problem with memory and arrays.

I have a program that runs for a long time (several weeks) but it seems to
gradually increase the size of the memory it uses. I have run it for 40 days
straight and the memory size increased from 1.7Mb to 66Mb. This program
populates an array with many elements, which it refreshes every hour, using
delete_data.

I have concluded that the garbage collection algorithm is either not working
or not cleaning everything up.

If you find a solution please let everyone know.

Thanks,
Ashley Stead
Anitech
Australia

"Roger Loftus" <rloftus@plantingstock.com> wrote in message
news:iSmHSBdbFHA.1436@dacmail.dataaccess.com...
> Glenn Hickman wrote:
>
>> Hi,
>>
>> We are having a problem with a program, it runs fine for a while but
>> then we get 'OUT OF MEMORY STATUS 10' error what causes this? and is
>> there a command to clear out memory we could maybe run whenever this
>> program is run? The program uses 15 arrays is this the problem?
>>
>> Best Regards,
>> Glenn Hickman
>
> Glenn:
>
> Wow - 15 data arrays? If these arrays accumulate a significent number
> of items, I would surely think that this is the source of your status
> 10 at runtime.
>
> You probably need to exercise control over these arrays if possible.
> If it is an option, sending delete_data to an array not currently
> needed frees the memory consumed by its contents. I believe also using
> delete_item on array elemements frees memory in a similar manner. You
> may need to experiment with writing an array out to a temporary data
> file and reading it back in to perserve needed array data but reduce
> concurrent array item counts.
>
> The only other option is probably to focus on maximizing the free
> memory of your session, and that is dependent on your OS and whether
> you are using DOS or console mode.
>
> Roger Loftus

Roger Loftus
11-Jun-2005, 02:58 AM
Ashley Stead wrote:

> Hi,
> I also have a problem with memory and arrays.
>
> I have a program that runs for a long time (several weeks) but it
> seems to gradually increase the size of the memory it uses. I have
> run it for 40 days straight and the memory size increased from 1.7Mb
> to 66Mb. This program populates an array with many elements, which it
> refreshes every hour, using delete_data.
>
> I have concluded that the garbage collection algorithm is either not
> working or not cleaning everything up.
>
> If you find a solution please let everyone know.
>
> Thanks,
> Ashley Stead

Hmmm, Ashley.

I just tried a small program to repeatedly load array objects to memory
limits, and test program memory useage along the way. For me,
delete_data seems to repeatedly recover every bit of the memory that
was allocated to array data in this test program. So, I just don't
know.

I think if I had such a robust need for array storage, I would be
tempted to set up a ram drive on the workstation and use standard file
commands to set and retreve data. I could then use make_file,
zerofile, find, save and delete to setup, empty, and add and delete
array (record)elements to the ram database array. This would provide a
ram speed data area completely separate from the memory used by the
runtime.

I think free downloadable ramdrive setups are available for almost all
Microsoft OS platforms.

Roger Loftus

Ashley Stead
11-Jun-2005, 03:04 AM
Roger,

I forgot to mention that I was using Dataflex 3.2 on Linux. I'm not sure its
the array but there is not much else the program does apart from checking
the array very often (every 15 seconds) and executing programs when it
checks the parameters in the array.

Ashley Stead
Anitech
Australia

"Roger Loftus" <rloftus@plantingstock.com> wrote in message
news:EuLJrtlbFHA.1276@dacmail.dataaccess.com...
> Ashley Stead wrote:
>
>> Hi,
>> I also have a problem with memory and arrays.
>>
>> I have a program that runs for a long time (several weeks) but it
>> seems to gradually increase the size of the memory it uses. I have
>> run it for 40 days straight and the memory size increased from 1.7Mb
>> to 66Mb. This program populates an array with many elements, which it
>> refreshes every hour, using delete_data.
>>
>> I have concluded that the garbage collection algorithm is either not
>> working or not cleaning everything up.
>>
>> If you find a solution please let everyone know.
>>
>> Thanks,
>> Ashley Stead
>
> Hmmm, Ashley.
>
> I just tried a small program to repeatedly load array objects to memory
> limits, and test program memory useage along the way. For me,
> delete_data seems to repeatedly recover every bit of the memory that
> was allocated to array data in this test program. So, I just don't
> know.
>
> I think if I had such a robust need for array storage, I would be
> tempted to set up a ram drive on the workstation and use standard file
> commands to set and retreve data. I could then use make_file,
> zerofile, find, save and delete to setup, empty, and add and delete
> array (record)elements to the ram database array. This would provide a
> ram speed data area completely separate from the memory used by the
> runtime.
>
> I think free downloadable ramdrive setups are available for almost all
> Microsoft OS platforms.
>
> Roger Loftus

Roger Loftus
11-Jun-2005, 11:42 AM
Ashley Stead wrote:


> I forgot to mention that I was using Dataflex 3.2 on Linux. I'm not
> sure its the array but there is not much else the program does apart
> from checking the array very often (every 15 seconds) and executing
> programs when it checks the parameters in the array.
>

Ashley,

I don't know anything about Linux, but if your Dataflex program is
spawning tasks using runprogram, are you sure the spawned tasks are
releasing memory to your Dataflex runtime environment, or adding some
un-released overhead at each calling? I would be tempted to look at
this as more likely than array data overload given that you are purging
the array regularly which should prevent the enormous memory
accumulation.

Is it possible you are chain waiting dataflex programs from the array
driven app instead?

Roger Loftus

Ashley Stead
11-Jun-2005, 03:00 PM
Roger,

I don't chain any tasks. I create a shell script that is suppose to run
asynchronously but doesn't. I use an edit object to collect the output using
runprogram pipe. Maybe this is the problem. I'll check when I'm back at
work.

Ashley Stead
Anitech
Australia

"Roger Loftus" <rloftus@plantingstock.com> wrote in message
news:Wz4jQSqbFHA.1276@dacmail.dataaccess.com...
> Ashley Stead wrote:
>
>
>> I forgot to mention that I was using Dataflex 3.2 on Linux. I'm not
>> sure its the array but there is not much else the program does apart
>> from checking the array very often (every 15 seconds) and executing
>> programs when it checks the parameters in the array.
>>
>
> Ashley,
>
> I don't know anything about Linux, but if your Dataflex program is
> spawning tasks using runprogram, are you sure the spawned tasks are
> releasing memory to your Dataflex runtime environment, or adding some
> un-released overhead at each calling? I would be tempted to look at
> this as more likely than array data overload given that you are purging
> the array regularly which should prevent the enormous memory
> accumulation.
>
> Is it possible you are chain waiting dataflex programs from the array
> driven app instead?
>
> Roger Loftus

Danny Berman
12-Jun-2005, 02:03 PM
I ran into a similar problem with a program that runs all day, and does
frequent runprogram waits. After a couple of hours or so, it would suddenly
be unable to execute the command (pkunzip, for example). I suspected a
memory problem of some kind, but the memory command always showed tons of
memory. Perhaps it had to do with file handles, so I tried closing files
before runprogram wait, and reopening after. No luck.

I never found the cause of the problem, but did come up with a workaround
that solves it, at least in this case: Once every hour, the program shuts
itself down, but before doing so it creates an empty file with a specific
filename (like restart.job). The batch file that runs the program checks
for the existence of this file and restarts the program if it exists. The
restart.job file is deleted as part of the startup of the program. It only
takes about 2 seconds to exit and restart the job, and has eliminated hours
of my staff waiting for the program to perform its functions and for me to
diagnose why things weren't happening. Understandably, some jobs can't be
stopped and started easily, and this shouldn't be necessary, but I'm happy
to offer the suggestion for anybody who can use it.

-Danny-

"Ashley Stead" <astead@bigpond.net.au> wrote in message
news:YIGHWCsbFHA.1276@dacmail.dataaccess.com...
Roger,

I don't chain any tasks. I create a shell script that is suppose to run
asynchronously but doesn't. I use an edit object to collect the output using
runprogram pipe. Maybe this is the problem. I'll check when I'm back at
work.

Ashley Stead
Anitech
Australia

"Roger Loftus" <rloftus@plantingstock.com> wrote in message
news:Wz4jQSqbFHA.1276@dacmail.dataaccess.com...
> Ashley Stead wrote:
>
>
>> I forgot to mention that I was using Dataflex 3.2 on Linux. I'm not
>> sure its the array but there is not much else the program does apart
>> from checking the array very often (every 15 seconds) and executing
>> programs when it checks the parameters in the array.
>>
>
> Ashley,
>
> I don't know anything about Linux, but if your Dataflex program is
> spawning tasks using runprogram, are you sure the spawned tasks are
> releasing memory to your Dataflex runtime environment, or adding some
> un-released overhead at each calling? I would be tempted to look at
> this as more likely than array data overload given that you are purging
> the array regularly which should prevent the enormous memory
> accumulation.
>
> Is it possible you are chain waiting dataflex programs from the array
> driven app instead?
>
> Roger Loftus

Glyn astill
13-Jun-2005, 05:52 AM
Should init_instance also unallocate all memory just like delete_data?
At the moment I'm doing both when I need to reset my arrays, then
destroying the arrays at the end of the program with destroy_object.

Roger Loftus wrote:
> Ashley Stead wrote:
>
>
>>Hi,
>>I also have a problem with memory and arrays.
>>
>>I have a program that runs for a long time (several weeks) but it
>>seems to gradually increase the size of the memory it uses. I have
>>run it for 40 days straight and the memory size increased from 1.7Mb
>>to 66Mb. This program populates an array with many elements, which it
>>refreshes every hour, using delete_data.
>>
>>I have concluded that the garbage collection algorithm is either not
>>working or not cleaning everything up.
>>
>>If you find a solution please let everyone know.
>>
>>Thanks,
>>Ashley Stead
>
>
> Hmmm, Ashley.
>
> I just tried a small program to repeatedly load array objects to memory
> limits, and test program memory useage along the way. For me,
> delete_data seems to repeatedly recover every bit of the memory that
> was allocated to array data in this test program. So, I just don't
> know.
>
> I think if I had such a robust need for array storage, I would be
> tempted to set up a ram drive on the workstation and use standard file
> commands to set and retreve data. I could then use make_file,
> zerofile, find, save and delete to setup, empty, and add and delete
> array (record)elements to the ram database array. This would provide a
> ram speed data area completely separate from the memory used by the
> runtime.
>
> I think free downloadable ramdrive setups are available for almost all
> Microsoft OS platforms.
>
> Roger Loftus

Glyn astill
13-Jun-2005, 07:41 AM
Well I've rechecked my program and it seems to loose memory every time
it goes around a loop, for each iteration it accepts an input and then
redisplays the information to the screen, it sends delete_data to the
array then init_instance to the array then puts each item it has
displayed to the screen in the array, it doesn't do any finds or saves,
nor does it runprogram or chain out.

From what I gather from doing the following in the loop I loose 324
bytes every iteration

memory my_mem
gotoxy 24 4
show my_mem

It is mad.

Any ideas people?

Regards
Glyn

Danny Berman wrote:
> I ran into a similar problem with a program that runs all day, and does
> frequent runprogram waits. After a couple of hours or so, it would suddenly
> be unable to execute the command (pkunzip, for example). I suspected a
> memory problem of some kind, but the memory command always showed tons of
> memory. Perhaps it had to do with file handles, so I tried closing files
> before runprogram wait, and reopening after. No luck.
>
> I never found the cause of the problem, but did come up with a workaround
> that solves it, at least in this case: Once every hour, the program shuts
> itself down, but before doing so it creates an empty file with a specific
> filename (like restart.job). The batch file that runs the program checks
> for the existence of this file and restarts the program if it exists. The
> restart.job file is deleted as part of the startup of the program. It only
> takes about 2 seconds to exit and restart the job, and has eliminated hours
> of my staff waiting for the program to perform its functions and for me to
> diagnose why things weren't happening. Understandably, some jobs can't be
> stopped and started easily, and this shouldn't be necessary, but I'm happy
> to offer the suggestion for anybody who can use it.
>
> -Danny-
>
> "Ashley Stead" <astead@bigpond.net.au> wrote in message
> news:YIGHWCsbFHA.1276@dacmail.dataaccess.com...
> Roger,
>
> I don't chain any tasks. I create a shell script that is suppose to run
> asynchronously but doesn't. I use an edit object to collect the output using
> runprogram pipe. Maybe this is the problem. I'll check when I'm back at
> work.
>
> Ashley Stead
> Anitech
> Australia
>
> "Roger Loftus" <rloftus@plantingstock.com> wrote in message
> news:Wz4jQSqbFHA.1276@dacmail.dataaccess.com...
>
>>Ashley Stead wrote:
>>
>>
>>
>>>I forgot to mention that I was using Dataflex 3.2 on Linux. I'm not
>>>sure its the array but there is not much else the program does apart
>>>from checking the array very often (every 15 seconds) and executing
>>>programs when it checks the parameters in the array.
>>>
>>
>>Ashley,
>>
>>I don't know anything about Linux, but if your Dataflex program is
>>spawning tasks using runprogram, are you sure the spawned tasks are
>>releasing memory to your Dataflex runtime environment, or adding some
>>un-released overhead at each calling? I would be tempted to look at
>>this as more likely than array data overload given that you are purging
>>the array regularly which should prevent the enormous memory
>>accumulation.
>>
>>Is it possible you are chain waiting dataflex programs from the array
>>driven app instead?
>>
>>Roger Loftus
>
>
>
>

Roger Loftus
13-Jun-2005, 09:51 AM
Glyn astill wrote:

> Well I've rechecked my program and it seems to loose memory every
> time it goes around a loop, for each iteration it accepts an input
> and then redisplays the information to the screen, it sends
> delete_data to the array then init_instance to the array then puts
> each item it has displayed to the screen in the array, it doesn't do
> any finds or saves, nor does it runprogram or chain out.
>
> From what I gather from doing the following in the loop I loose 324
> bytes every iteration
>

Glyn:

Try leaving out the init_instance call and try your program again.

Roger Loftus

Glyn astill
13-Jun-2005, 10:06 AM
Hi Roger,

I will try this. Why do you suspect the init_instance is the problem?

Regards
Glyn

Roger Loftus wrote:

> Glyn astill wrote:
>
>
>>Well I've rechecked my program and it seems to loose memory every
>>time it goes around a loop, for each iteration it accepts an input
>>and then redisplays the information to the screen, it sends
>>delete_data to the array then init_instance to the array then puts
>>each item it has displayed to the screen in the array, it doesn't do
>>any finds or saves, nor does it runprogram or chain out.
>>
>> From what I gather from doing the following in the loop I loose 324
>>bytes every iteration
>>
>
>
> Glyn:
>
> Try leaving out the init_instance call and try your program again.
>
> Roger Loftus

Roger Loftus
13-Jun-2005, 10:49 AM
Glyn astill wrote:

> Hi Roger,
>
> I will try this. Why do you suspect the init_instance is the problem?
>
> Regards
> Glyn
>

Glyn:

I am pretty sure using init_instance to reset your array is a misuse of
its intended purpose. Once the array is activated, delete_data will
purge all items form your array, release the memory used, and should do
everything you need to do to start filling the array again. I am not
sure, but init_instance may be setting aside memory for an object
structure every time it is called. I may run a trace on the call out
of curiosity just to see what happens. In any case, it is the most
suspicous part of your program to me, and I think I would try removing
init_instance to determine if it is the problem or not. I am at least
pretty sure that your program will function without it.

Roger loftus

Glyn astill
13-Jun-2005, 12:41 PM
Roger,

Thanks for this.

It seems to be working so far, the memory goes up and down rather than
just down all the time.

I'm failing to see the point of init_instance at the moment.

Glyn

Roger Loftus wrote:
> Glyn astill wrote:
>
>
>>Hi Roger,
>>
>>I will try this. Why do you suspect the init_instance is the problem?
>>
>>Regards
>>Glyn
>>
>
>
> Glyn:
>
> I am pretty sure using init_instance to reset your array is a misuse
of
> its intended purpose. Once the array is activated, delete_data will
> purge all items form your array, release the memory used, and should
do
> everything you need to do to start filling the array again. I am not
> sure, but init_instance may be setting aside memory for an object
> structure every time it is called. I may run a trace on the call out
> of curiosity just to see what happens. In any case, it is the most
> suspicous part of your program to me, and I think I would try removing
> init_instance to determine if it is the problem or not. I am at least
> pretty sure that your program will function without it.
>
> Roger loftus

Ashley Stead
14-Jun-2005, 02:55 AM
Roger,
I had a close look at when the memory was being allocated and I found that
it could be anywhere, particularly if you call procedures and function.
Therefore I am suspicious of the stack allocation process not releasing
memory that is no longer used.

Ashley Stead
Anitech
Australia

"Ashley Stead" <astead@bigpond.net.au> wrote in message
news:YIGHWCsbFHA.1276@dacmail.dataaccess.com...
> Roger,
>
> I don't chain any tasks. I create a shell script that is suppose to run
> asynchronously but doesn't. I use an edit object to collect the output
> using runprogram pipe. Maybe this is the problem. I'll check when I'm back
> at work.
>
> Ashley Stead
> Anitech
> Australia
>
> "Roger Loftus" <rloftus@plantingstock.com> wrote in message
> news:Wz4jQSqbFHA.1276@dacmail.dataaccess.com...
>> Ashley Stead wrote:
>>
>>
>>> I forgot to mention that I was using Dataflex 3.2 on Linux. I'm not
>>> sure its the array but there is not much else the program does apart
>>> from checking the array very often (every 15 seconds) and executing
>>> programs when it checks the parameters in the array.
>>>
>>
>> Ashley,
>>
>> I don't know anything about Linux, but if your Dataflex program is
>> spawning tasks using runprogram, are you sure the spawned tasks are
>> releasing memory to your Dataflex runtime environment, or adding some
>> un-released overhead at each calling? I would be tempted to look at
>> this as more likely than array data overload given that you are purging
>> the array regularly which should prevent the enormous memory
>> accumulation.
>>
>> Is it possible you are chain waiting dataflex programs from the array
>> driven app instead?
>>
>> Roger Loftus
>
>

Roger Loftus
14-Jun-2005, 07:40 AM
Ashley Stead wrote:

> Roger,
> I had a close look at when the memory was being allocated and I found
> that it could be anywhere, particularly if you call procedures and
> function. Therefore I am suspicious of the stack allocation process
> not releasing memory that is no longer used.
>
Ashley,

Reviewing your posts again, it is pretty evident that your program sort
of pushes the envelop on memory data handling, and depends on a rather
constant interaction between Dataflex and other called programs. Given
that, it is pretty hard to suggest places to look to solve your memory
issues.

Let me comment on the above as it relates to a running Dataflex
program, and stress that I am not the best authority on the subject. I
do not think you should lose memory calling procedures and functions
unless these calls add data to edit, list, or array based objects.
Maybe if you test memory within a procedure you might detect the
allociation for local variables in the procedure if they were
significent, but these should be released completely when the procedure
completes. Other than that, I think memory is allociated when you
start the UI, and there should be no cumulative memory effects of
operating within UI other than data collection as mentioned.

I think there are a few places to go wrong that you might check.
First, be sure that you are not starting multiple UI levels using
ui_accept or start_ui, and that UI levels are not inadvertantly
accumulating in your program. Second, be careful of procedures that
declare objects as you may be accumulating objects as your program
runs. Finally, be careful if you are relying on destroy_object as I
think you must use it properly (destroy created objects in proper
sequence) to recover the memory used.

I don't know if any of this applies to your program, and I would still
be most concerned about effects of the runprogram interaction.

Roger Loftus

Ashley Stead
15-Jun-2005, 02:59 AM
Roger,

I have given up trying to solve this. In Linux the memory command always
return the largest number possible. I have tried using a special file in the
/proc directory which has the current memory usage for my process, but it
only increments in 4k blocks. The increase in memory seems to happen
anywhere but sometimes it actually reduces, but overall it still increases.

My work around is to put my program in a shell loop testing to see if a file
exists which is created when my program wants to continue but its time is
up. If the file is not there it drops out.

Thanks,
Ashley Stead
Anitech
Australia


"Roger Loftus" <rloftus@plantingstock.com> wrote in message
news:5LOXE5NcFHA.1276@dacmail.dataaccess.com...
> Ashley Stead wrote:
>
>> Roger,
>> I had a close look at when the memory was being allocated and I found
>> that it could be anywhere, particularly if you call procedures and
>> function. Therefore I am suspicious of the stack allocation process
>> not releasing memory that is no longer used.
>>
> Ashley,
>
> Reviewing your posts again, it is pretty evident that your program sort
> of pushes the envelop on memory data handling, and depends on a rather
> constant interaction between Dataflex and other called programs. Given
> that, it is pretty hard to suggest places to look to solve your memory
> issues.
>
> Let me comment on the above as it relates to a running Dataflex
> program, and stress that I am not the best authority on the subject. I
> do not think you should lose memory calling procedures and functions
> unless these calls add data to edit, list, or array based objects.
> Maybe if you test memory within a procedure you might detect the
> allociation for local variables in the procedure if they were
> significent, but these should be released completely when the procedure
> completes. Other than that, I think memory is allociated when you
> start the UI, and there should be no cumulative memory effects of
> operating within UI other than data collection as mentioned.
>
> I think there are a few places to go wrong that you might check.
> First, be sure that you are not starting multiple UI levels using
> ui_accept or start_ui, and that UI levels are not inadvertantly
> accumulating in your program. Second, be careful of procedures that
> declare objects as you may be accumulating objects as your program
> runs. Finally, be careful if you are relying on destroy_object as I
> think you must use it properly (destroy created objects in proper
> sequence) to recover the memory used.
>
> I don't know if any of this applies to your program, and I would still
> be most concerned about effects of the runprogram interaction.
>
> Roger Loftus
>