IT Employment

The problem with learning Big Data in school? Not having any data

The biggest stumbling block for those teaching these classes and those wanting to learn is little to no access to large, relevant data sets.

Unless you've been living under 12 feet of ice in the Antarctic Plateau, you know that Big Data is the latest IT buzz term. IT careerists all over are advising folks get on the Big Data wagon.

Colleges are scrambling to create programs in business intelligence and Big Data analytics for those who want to work in a world overrun by data. But how's this for cruel irony? The biggest stumbling block for those teaching these classes and those wanting to learn is little to no access to large, relevant data sets.

It makes sense. What entity is going to hand over its data to be experimented on? The findings come from a recent survey of professors and students (professors at 243 universities in 43 countries and students at 96 universities) conducted by Teradata and Barb Wixom, an associate professor at the University of Virginia.

If you're thinking about taking classes in order to do data analysis in an IT/discipline, it might be a good idea to ask your advisor specifically about the availability of data to work with.

About

Toni Bowers is Managing Editor of TechRepublic and is the award-winning blogger of the Career Management blog. She has edited newsletters, books, and web sites pertaining to software, IT career, and IT management issues.

8 comments
dogknees
dogknees

"Pursuit of professional advancement as one's chief or sole aim:" is the definition of a "careerist", so why would anyone take any notice of someone like this? They are far too self centered to be any kind of authority on where IT should be heading.

gcrook
gcrook

Two thoughts, Toni where did you get the data that supports your data not available hypothesis? And for those interested in exploring Big Data - watch for courses on Coursera.

rduncan
rduncan like.author.displayName like.author.displayName 2 Like

Public data sets are freely available, Google have 103 sets - first search result returned http://www.google.com/publicdata/directory ..ask you local weather station, get cenus data from around the world, make an FOI request- I think perhaps you meant hosting your own large data sets on fast storage is the problem - use the cloud!

yhk331
yhk331 like.author.displayName 1 Like

I just want to let people know that you do have access to big data. For example, Kaggle offers competition which comes with large amount of data.

ahanse
ahanse like.author.displayName 1 Like

Better to check the credentials of those teaching...

Slayer_
Slayer_

When I was working with big data, for test systems I wrote an SQL script that read text files of names and stuff. And made up data. Then I just let it run for a few hours, creating a couple million records.

Archives Steve
Archives Steve like.author.displayName 1 Like

I agree with dbmarketing. The Federal government has lots of big data sets available for the asking. Many are published on data.gov. The National Archives has more.

dbmarketing
dbmarketing like.author.displayName like.author.displayName 2 Like

When I was in my political science program we pulled data off of the census results continually, and compared it to data from other federal and state programs. If millions upon millions of records isn't big enough for ya, I can't help you :)

Editor's Picks