How can we manage data growth?
Planning storage and infrastructure for ever-larger data volumes is a challenge many businesses face. It helps to put the scale in context.
IDC forecast that the global datasphere would reach 163 zettabytes by 2025, ten times the 16.1 zettabytes generated in 2016. Source: IDC white paper by David Reinsel, John Gantz and John Rydning, Data Age 2025, sponsored by Seagate, April 2017. View the paper
Data management for data growth
Is your data structured or unstructured? The answer shapes how you store, govern and move it.
Structured data
Typically information held in a database. It can be easily entered, stored, accessed and queried.
Unstructured data
The main problem for businesses today. With no form of structure, everything from social media posts, business documents, email, images and texts raises the same question: where do we store it?
Identification: know your data first
With so much unstructured data arriving, it needs to be managed in a way that answers all of the following.
Value and ownership
- How valuable is this information?
- Does the data have intellectual property?
- Who has access to it?
Governance and legal
- Does it belong on the business network?
- What rules do we have around data retention and deletion?
- What are the legal requirements on this information?
- Can individuals copy data from the business network to services such as Dropbox?
Location and copies
- How many copies of the data exist?
- Where is this information stored?
- Can you move the data to something more affordable?
Growth and tooling
- What applications are driving the data growth?
- What issues do you face in handling ever-increasing data volumes?
- What plans have you made to streamline it?
- What existing data analysis tools do you have to investigate and mine the data?
Where to store it?
Data growth is today the biggest challenge for even the best IT managers. The diagram below shows the various storage tiers. Do you have the tools to move data between the tiers automatically, or do you move information by hand when required? See our guide to tiered storage for more.
What each tier costs you
- Every system shown, with the exception of paper, consumes energy and, depending on the tier, costs a small or large amount of money.
- Putting data in the cloud costs money: you pay every month to store it, replicate it and download it, and it is slow to upload and download compared with on-site storage. See the challenge of predicting cloud costs.
- Growth has to be managed and controlled. Write procedures for how long data is kept or deleted, and involve Legal and HR in the corporate governance of the data.
The problems
Nobody wants the job
Like backup, this is a task no one likes to discuss or get involved with. It has normally been neglected and is now a bigger problem than anyone envisaged.
It takes time
Getting to grips with data growth is not easy, and it could take months or years to get working right from the start.
Tool choice matters
Choosing the right technology and software is not for the faint-hearted, and it could cost many thousands to get started.
Streamline data growth
Your most expensive storage shouldn't be used to store legacy data.
With data growth continuing to accelerate, rules and policies should be implemented to move data automatically to the most appropriate location, with little or no user input. That could include migrating data to network attached storage (NAS), a file server or the cloud. Our guide to data archiving covers the long-term side.
Managing data growth effectively can give you:
- Visibility into where your data resides, who owns it and how old it is.
- Better-performing systems.
- Significantly reduced backup volumes.
- Savings from extending the lifespan of your existing storage investments.
Talk to Fortuna Data about your data growth
We can advise on, and provide, a range of solutions to meet your business requirements and expectations. Speak to one of our specialists.
Prefer email? Write to solutions@fortunadata.com.