Question Locked

100gb of data, which database?

0 helpful
3 replies
Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.

I will be purchasing the data contained in OJIN(Oregon Judicial Information Network) from The State of Oregon. Everything which has occurred in Oregon State Courts since the late 80?s will be contained. The data will be sent to me on 27 DVD?s, so in excess of 100 gigs of raw data in a fixed length format, will be arriving. They sent a sample data file and 2 Excel files which explain field names and lengths. Once the database has been created, it will NEVER BE UPDATED.

I really need answers to the following:

1. What type of database should I focus on? Is there a desktop database I can buy and then find someone proficient to build it?
2. What can I expect to pay for this database creation?
3. Can the work be accomplished remotely?
4. How long will the job take?

I realize that my ignorance is glaringly obvious, can someone enlighten me, please?

Steven K. Henry
North American Solutions, LLC
P.O. Box 125
Lake Oswego, OR 97034
503-675-6485 voice and fax

3 replies

by rkuhn040172

First off, it's important to keep in mind that just because the data files are 100GB + that doesn't mean the DB will be.

Unless those data files are being compressed while on the DVD's, I'm pretty sure any DB will do a much better job storing the data than the files themselves.

Second, not knowing about all the alternatives I will say that I doubt a desktop DB will be appropriate. For example, MS SQL Express I believe is only good for up to 4GB. Most other competitors will probably be similar.

That said, unfortunately you are probably looking at the need for a server and a real business size/quality DB. $$$

You could probably save a lot of money if you went open source but then support and what not might be less readily available.

That being said, a server and something like SQL Server isn't going to be cheap!

by tony hopkinson

From the sound of it you are going to contract out the conversion from the raw data to a database. You might want to leave the choice down to whoever you get in to do the job.

Converting from X to Y is a fairly trivial task compared to building the thing in the first place, so even if you don't go with what they use they can give you the processed files in a format you can import to almost anything.

As for how you should implement the final version. Too many variables.
Who is going to host it, how is it going to be accessed?. What sort of usage is it going to get. How powerful are the interrogation facilities going to be, googlising the transcripts for instance.

You need to answer some of these questions first.

This discussion is closed to new replies.