Question Locked

Anyone familiar with System.OutOfMemoryException ?

0 helpful
18 replies
Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.

I've googled, and found some interesting bits but a lot of them relate to MS Reporting Services, BizTalk etc.

Basically, have a C# application accessing a SQL database. For months now we've been getting the: System.OutOfMemoryException and the web app crashes. I just go and reboot the web server and its fine.

Now, when this first started happening, it was happening every 10-12 days without fail, but now it seems to be a bit more random. Will happen 3 times in one day, then be fine for 4-10 days. Then 3 days in a row, then nothing for a week.

Our hardware company (outsourced)maintains that there is nothing wrong with the server, there is plenty of memory, and it should not be happening, and maybe its a memory leak in the application and our software guys should check the app.

Our software company (outsourced) maintains that there is a problem with the memory and our hardware guys should fix it.

Meanwhile, I'm stuck in the middle. Has anyone had any similar experiences?

18 replies

by tony hopkinson

https://blogs.msdn.com/yunjin/archive/2004/01/27/63642.aspx
or more correctly your softwsre supplier.

System.OutOfMemory has nothing to do with hardware, except in this sort of situation it will get raised more if you have less.

Unmanaged objects was raised as a possibility, you don't happen to know if the c# version was a port from C++ do you?

If it was that's the first place I'd look. The garbage collector was invented becasue most C++ coders are outright crap at lifetime management.

by shellbot

i'll go have a read of that this morning!

Well, i've been maintaining that it is not a hardware problem, and basically the development company can't be bothered to go check the app

I'm pretty certain the code was actually written in C#, as its a bespoke application. So unless they had a similar app they just ported over and rehashed..? I wouldn't put it past these guys.

From Jan 1,2005 to June 2006, this problem had never occured..in May we got a bit of work done, enhancements, bug fixes etc..and about 3 weeks later this started happening.

Unfortuneatly I'm only hired as a DBA, so my boss seems to think that my programming skills are not required to resolve this issue..no matter how many times i put my hand in there air, scream "over here..i might know the answer" my comments and attempts to resolve the issue are overlooked..(long standing issue with employer)

Off i go to read..
thanks :)

by shellbot

I should have grabbed a cup of coffee before digging into that..

I get it though, not to say I'm even close to completely understanding it..but i get the concept.

I have a feeling if i send this onto the dev company they will look at me like I'm an alien!! But all i can do is try

by tony hopkinson

Even if the machine was struggling for physical ram, all that would mean is it would hit the swap file. My BS filters went into turbo mode as soon as I read that ah justification.

Brown dandruff everywhere, I despise people who take the urine like that. If it was a hardware fault why would it only affect their code? I'd expect to see an event log riddled with warnings and faults.

by ricardol_limon

that you have a memory leak, like Tony said. But I bet and I'm 90% sure that the developers didn't closed references to unused objects or dispose the database reference objects and those are a good way of building an OutOfMemory Excecption, how do i know? ummm lets say i learned the hard way...

Good Luck!

P.S. Who is bringing the coffee ?

by shellbot

i agree with the mem leak..it seems to be the consensus of everyone i ask.

I've a list of things i will never do again having learnt it the hard way :)

Thanks!

I'll bring coffee :)

by shellbot

well i'm really not sure what to do..because my boss(s) want to bring in some company with this huge server to do a test to see if ramping up the machine will do any good.

i've given my opinion that obviously it would help in the short term, maybe i'm wrong, but if its problem is in the application, its not going to make a whole lot of difference, because it will only eat up the extra memory anyways..ok, maybe i'm not right on that, but its treating the symptom, not the cause.

Funny thing, these guys who developed the application, i have asked for the source code, code behind and or solution files right (they were susposed to hand it over months ago)..yesterday they told me "its all there, on the server" I asked did they have a copy..nope..its all there on the server..so the voices in my head are saying "WHAT..the only copy of the solution is on a LIVE server..holy..."
So after talking with them, politely, i realise they don't know what i am talking about..when i said i was unable to find it, they directed me to the wwwroot folder.. "see all the asp pages are in there"

So i casually ask about the .dll in the bin, and the guy says, oh you don't need to know that code, thats just an include file to make the application work. You wouldn;t be interested in that, its all techie stuff..

?????????
I told him I'd get back to him..need way more coffee before i deal with this chap again. He's a web developer..and SQL developer I'm told...

This week won't be over quick enough

by hal 9000

But I think that you'll find yourself bashing your head against a brick wall and if your not you should be as it's less painful than attempting to deal with Ijots like this one.

Why would you need all that Technical Stuff and more importantly how do you know about it? :D

Because of where you are I would let your Boss solve the problem a Massive Server something like a 2,000 CPU Blade with unlimited Memory should cure your problem permanently though it's a Tad Expensive but if your Boss tells her Boss that it has to be done you're out of the picture and more importantly not in the firing line.

But I think that you'll find Tony is correct this C thing will be based on a C++ piece of Crap and pulling your system down as it leaks at an intermediate rate causing system crashes without warning or even showing a single Event in the Logs to attempt to explain what is happening.

I'll bet you $ to Donuts that the problem will eventually be found in some Compiled Code that's if it's ever traced down properly and not just have the symptom treated.

[b]OH My God What Have I done?[/b] I know absolutely nothing about programing at all and I've never heard of C or any thing like that Honest!!!!!! Now what does DLL actually stand for? I seem to remember something about [i]Do Ladies Listen[/i] is that it? :^0

Col

by shellbot

You know sometimes when you've smiled that big polite smile for ages and you just can't anymore and you just want to ram your monitor down..or up somewhere..?

thats the stage i was at when with these chaps.. i guess looking back, how silly of me to question them when they were explaining that the ASP pages ARE the source does, and that DLL file is just a bit of techie stuff...i'm just a silly girl i guess

causing system crashes without warning or even showing a single Event in the Logs to attempt to explain what is happening.

Thats it in a nutshell..i'm going to look further into this..been looking at a few of the things they've done, and some of it looks like they've got a C programmer..

But what do i know..
:)

by tony hopkinson

but I'll stay polite and say inexperienced.
"All techie stuff", impressive customer facing skills I must say.

I'll wager a doughnut along with Col that the dll leaks like a sieve and at least some of it was not written in C#.

Do you have a test environment with this on.
You might get some joy out of monitoring IIS's memory usage while calling up pages which use the dll.

by shellbot

and raise it 2 more..

Yes, impressive customer facing skills. I must admit though, it takes skill to compile a team of completly inadequate staff members. These guys are incredible..

They been working for the last week trying to fix an export (to a .csv) that keeps timing out..funny thing is, i told them where its going wrong..and they still can't sort it..but i guess thats some of that "techie stuff" that i shouldn't know :)

No test env at min, but will have soon, so i think i'll set aside a few days watching it :)

I think you guys are right though..its the direction i was thinking in, so its nice to see that i'm probably not completely off base!

by hal 9000

All that Coffee that you've been drinking and it's having an adverse affect on you as you are beginning to think that you know more than the company who wrote this mess. Worse still you are now suffering Delusions of Grandeur everyone knows that some silly woman couldn't possibly know more than the company who made the mess in the first place. :p

Odds are that they where a bunch of 20 somethings who have either all moved on to better paying jobs or are currently in the Loony Bin suffering from multiple Nervous Breakdowns.

Everyone knows that a C Compiler gives much better results to someone who can actually write the code and fault find it after all everything is now Automatic so it must be better right? :D

OH I just could have done so much more if there where C Compilers around when I was writing in that Language nd the code would have been so much better. [b]Sarcasm Alert Here[/b] I've yet to see some computer compiled code that works anywhere near as well as when someone actually writes it as they find every problem before they release it into the wild because it tends to break when it's tested. Those Nasty Compilers are fast and easy for what pass as Programmers today but they certainly leave a lot to be desired when it comes to writing decent code.

I never minded learning from my mistakes and God only knows that I've made plenty of them but I've never yet inflicted any of my mistakes on some other user who is charged with supporting what I wrote. Before I walked away from Coding I was the one who was constantly wadding through thousand lines of code to find out that, that new Subroutine that was inserted after being fully tested caused intermittent problems that where Mongrels to track down. But at least it was all In Hose Stuff so no one else suffered and I got to show the New Beaut Programmers when they had messed up. Now if only they had of told me the truth originally I could have found the problems so much faster but as they have something run for a week without problems that means that's it reliable under all conditions.

But you're up against it here no Source Code No ability right at the moment to run on a test machine and hammer the crap out of it while running monitoring programs that will slow down a production environment to a snails pace. Glad that it's you and not me as I would be wanting to kill someone by the end of the first day.

Personally I wouldn't even consider shoving a Monitor down someones neck as that not anywhere near painful enough but if there was a 747 handy that would be a different matter, but I'd drop the tank drain plugs out as the wings go in. :^0

My Best Wishes here as it's not something that I would be looking forward to try to [b]Fault Find![/b]

But if this makes you feel any better I've wasted the better part of 2 days messing around with a copy of XP that WGA broke and MS insisted that it was repairable so I didn't need to perform either a Wipe & Load or an In Place Install as that was a waste of time and effort but it's perfectly acceptable to spend 2 days making it work when just running an In Place install would have taken about 1 hour. But what I found totally unacceptable was 5 Broken DLL's where not treated as a problem caused by the faulty WGA Code and after I had that sorted I had to wait all over again to speak to someone else at MS to fix IE, Outlook and the Address Book all of which where no longer working as well as a few others that where thrown in just to keep me on my toes.

I'll know better next time it happens. :(

Col

by shellbot

addicted..and its giving me the jitters :)
I've cut back on the strength tho..does that help? (down to 1.5 heaped spoons of instant with 3 sugars..was up to 2.5 coffee and 4 sugar a while back)

anywho's..the delusions..its the voices in my head that lead me astray :)

This silly woman was on the ground laughing this morning when i asked them what the ASP timeout was set at..and they didn't know what that was (again..web developers..should they not know this?).
So i sent them screen shots after i went and checked myself. They were greatful..

As a note, someone (NOT me) has the timeout set to 200000 seconds..i'm wondering would that contribute to the problem if there was a hang up in the code, and its not being let go?

Dunno..its friday and all i can think about is getting outa here.

"leave a lot to be desired when it comes to writing decent code"
I've been called "anal" in my coding practices..best compliment I've ever recieved. Not saying i'm perfect, and i've got a LONG way to go before i would consider myself experienced, but at least what i do write stands up for itself and is clearly documented.

Well, i am close to killing someone, but at the end of the day, its all experience right? What does not kill me will make me stronger :)

"2 days making it work when just running an In Place install would have taken about 1 hour"
Ouch..i feel your pain. Even worse..having to call MS :) I've never had the pleasure myself, but some peers have told me some great stories :)

Good luck with it!!!

by hal 9000

Seems just a TAD on the extreme side to me, but there must be a reason for something set to that figure. Maybe they liked all those Zeros I can't see any reason to have it set so high unless it's set that way just because it's possible. :D

Maybe you should leave it alone for over a week when it falls over and see if it works all those hours latter it would be an interesting exercise just to see what happens. But do it on the Production Box and not a Trial Box as there is no fun if others don't get to see you testing the system. :p

But importantly Shell did they understand those Screen Dumps that you sent them? I doubt it as most of these new people don't understand so they react brashly when asked some obvious question.

As for the Coffee that's way too much you have to cut back for your own health when you can walk past the coffee machine without having to stop and get a cup for several years you are part way cured. As I'm pure and have never drunk the stuff I'm lucky though I'm hopelessly addicted to mints and just one brand which I'm constantly sucking on all the time. I'm now up to over a pack a day and requiring more so I'm not one to talk. :(

However I can relate this story to you. [i]Long Long Ago In a Galaxy Far Far Away[/i] at trade show here that runs for 10 days I was sent over to setup the main section of the place and do all the little odd jobs that required doing. But as I don't drink Coffee I wasn't allowed anywhere near the Coffee Machine when it was setup. The conscientious of opinion was that as I already hated that stuff I would do something that would make the coffee taste foul. Anyway over the 10 days of the show many people came to the stand and on a few occasions when I was there every one was commenting on the flavor of the coffee and just how good it was. To make matters worse they would hand this stuff out to anyone who asked and not just the people working the Display. Anyway at the end of the show as they where pulling everything apart [i]Naturally the Coffee machine was the last thing to be removed[/i] I had finished packing up all my hardware when suddenly there was a mass exodus from out the back of the display with everyone rushing off to be sick. Apparently on the first night when they where setting up the stand they had left the top of the water container open and a rat had got in drowned and had stewed there for the entire 10 + days. That's what gave every cup of coffee that they handed out that Unique taste.

After that I got one of those Rubber Rats and glued it to the top of the machine.

MS Tech Support are your [b]Friends[/b] or should that read [b]Fiends[/b] not quite sure which but I have tried to avoid them since the DOS Days when I got an error message that [b]Device 6[/b] wasn't fitted after loading MS DOS whatever. That was fairly confusing as after checking that everything was where it was supposed to be [b]Silly Me[/b] I decided to try to find out exactly what Device 6 is supposed to be. It wasn't listed in any of the manuals of the time nor in MS On Line Help Section so I had to [b]Bite the Bullet[/b] and ring MS. I kid you not 3 hours latter I was informed that Device 6 is the CPU but some very officious MS Tech Support Guy who totally ignored the fact that the unit would perform the POST start to load DOS and then fall over. Apparently I was so stupid that I had forgotten to fit a CPU. I just had to love the condescending way that I was told this and the complete silence on the phone when I asked [b]Well If I Didn't fit a CPU how do I get a Image on the Monitor and can alter the BIOS?[/b] I still don't know what Device 6 was in this case but a reinstall cured the problem and it worked perfectly without any hardware changes. Now if only I could get a computer to work without a CPU I could save so much money with every one that I built. :^0

Anyway have a Great Weekend and enjoy yourself. I'm going to a construction Site to see what they have mucked up this week while I've been away. Knowing the electricians working there they will now have scrapped all the Optical Fiber Cable that I left up there and replaced it with Phone Wire that they have supplied as when I caught them trying to stretch the cable they insisted that it was OK as they did things like this all the time.

I'm beginning to wonder if it was such a good idea to have some one else pull the Network Cable as this place is being built. They'll probable nick the Optical Fiber Cable and try to work out what it is used for and try pulling something else through or better still try to find something cheaper to pull for my needs. :D

Col ]:)

by madestroitsolutions

And that answer is precisely what the name of the error is. Your machine is out of memory and it has nothing to do with the physical aspect of it. Their software is not releasing resources and it eventually builds up to the point where everything starts to freeze and the machine chokes.
The intermittent occurrences are due to the changing workload on the server.
On another note though, and not to be disrespectful in any way, I agree with your manager, you shouldn't be looking into this :)
Drop this and let them figure it out. I think your expertise can be better utilized in other areas. If you still insist though, you can set up some performance counters in the machines to show them where the memory is going and take a few snapshots of the task manager window, which will most likely show the worker process holding tons of memory.

Good luck!

by raven_tl

If you get a crash you can get a dump of the memory by using ADPlus or DebugDiag and analyze it with WinDBG. It is alot easier if you have the symbols (.pdb files). Even if you cannot get the exact cause you will find what process (dll) is causing the problem so you have some more information.

http://support.microsoft.com/kb/286350

by chandru.pal

Since it is a C# app and you might have pulled out the records and cached it in the dataset. When the compiler encounters the line where the DataAdapter fills Dataset (Ex : da.Fill(ds)), at this line it will through the System.OutOfMemoryException.

The solution is try to read using DataReader so it will not cache the records in memory.

Or have a local mdb file and read and write the records into local mdb and use it for reporting purposes.

This discussion is closed to new replies.