Skip to main content

What does Big Data and ITER have in Common?

I work with Big data, I do not work with ITER.  But both appear to suffer from the same condition, they both promise great benefits in the future.

Age does provide some benefits, with perspective being one of them.   For the last (well as long as it's been thought about), un-teen years, the promise of harnessing fusion reaction as an energy source has always been 20 years out.   If we just put our minds to it, we can have it done in about 20 years.  Well, we have been putting our minds to it and it's still 20+ years in the future.  By putting our minds "to it" means, a great number of countries have pledged people and resources to the ITER project that is currently being built in France (starting in 2007).  It's budget is now grown 3x times (16b) and the end is not in sight.

The hope, the promise, the future is always just out there, but not yet in our grasp.   It appears that harnessing fusion is way way harder than just creating un-controlled fusion.

On to Big Data.

In the IT world, we can now collect (and have been collecting) big data.  The capacity  and low cost is there and the software (Hadoop, ElasticSearch, etc) now exists for accessing and processing it in a timely fashion.

Life is good.  Except,  collecting data is not the same as harnessing and controlling it.

The expectation is that once you have all of this data, you can mine it for valuable information that you can 1) sell, 2) make your existing product / service better or 3) something new.  Ah, the promise of what the future will provide.  I'm waiting.....

At least with ITER there are 1) blueprints, 2) physics and 3) specific goals in place (even if it's all very bleeding edge stuff).  With big data, it's still all very mushy as to what you can do with it and how it should work with your existing data set.  It's all very much up in the air.  Just having a huge amount of data and some tools does not a solution make.

Collecting large amounts of data can work for targeted issues as long as the collected data is relevent and the methodologies are known for that domain.  But this is not what is being sold.  The storage and tools are being sold with no insight on what the solution is.   There is even a push that garbage data will be irrelavent as it will be drowned out by the patterns that arise.  Except when the patterns are the garage data (think Google Flu).

Big data may be the future, but we live in the here and now and solutions need to be delivered today.




Comments

Popular posts from this blog

My Current Desktop and Happy Vining.

It's been a big switch when I quit my old procession (Software Engineer) and ventured into the unknown.  So this is my current desktop.   First it's a REAL desktop and not a computer screen. Second, I've had to learn a whole new set of tools.  You would think a pencil is just a pencil and a paint brush is just a paint brush, but noooooo, they are tools into you mind and that's what needs re-tooling. The tactual nature of these tools and operation as much more rewarding (and frustrating too) than typing code in an IDE.   That had it's incentive but that well went dry and life changes,  so I changed. The pleasure of creating colors (no RGB involved) ad hoc wise is interesting.  Getting the technique to merge them on paper is very tough.  You do not know what you're going to get, so you just have to go with it. I'm glad that I can do a lot of other things instead of Software, but I hope that others in my past procession work on other pa...

3rd Try is a Charm

I've been trying to draw / paint these barns for a couple of years but never felt or got them right.  This time I think they turned out right. So What went wrong before and what's right now with this drawing?  This time, the light was right.  It's coming from the upper right and the shadows just looked right.  The other thing is the corn field on the left had to "be in season", otherwise it's just a plowed field.  I had taken other photos from different angles but they never felt right.  This angle has the road, power lines, corn field, etc. all leading to the right.  The shadows on the lower right helps fill in that corner (don't forget about the corners!).  The last part is trying to draw (ink paint maybe) the trees in the background.  Not so easy when they are kind of a blob is green shades. So yeah, it's composition that is king.  Many times I just don't see it until the drawing / painting is finished and when it's right it feels goo...

Thought Patterns

I know how I work.  How a think about, visualize a problem and go about mentally design a solution and implement it in code.  My thought pattern works pretty well for me on most occasions, but not always. There are times when I have a force myself to change my thought pattern because of the problem I'm solving requires a different mindset.  Anyway, this posting is about how I work most of the time.  Here is goes. I quickly break down a feature into operations that are required to exists and tackle that code first.  My thinking is that, if I can solve the primary issues first, then secondary features will fall into "known knowns" and I can address those at anytime (and sometimes at my leisure) and this allows me to meet my internal timeline. If I addressed features and operations that are not primary first, then I feel I risk my (and the projects) timeline without addressing issues that allow the feature to operate.  You may have seen this occur where...