DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB
Hot Topics
  • Where Does Observability Stand Today, and Where is it Going Next?
  • Five Great DevOps Job Opportunities
  • A Freelancer's Workflow
  • Azure Migration Strategy: Tools, Costs and Best Practices
  • Blameless Integrates Incident Management Platform With Opsgenie

Home » Blogs » Unstructured Data Management: Avoiding Insider Knowledge Gaps

Unstructured Data Management: Avoiding Insider Knowledge Gaps

Avatar photoBy: Scotty Calkins on August 19, 2022 Leave a Comment

Most IT leaders know that managing an unstructured data environment can be quite an undertaking. Before beginning an unstructured data management project, there are a lot of factors to consider including the number of storage devices, the size of a particular dataset, the number of files in the dataset and, perhaps the most intimidating, the unique environment the unstructured data inhabits.

One-size-fits-all solutions can often inadvertently sweep important characteristics of unstructured data environments under the rug. Granted, this short-term, simple solution is easier than putting in the work to customize the best-fit solution for each organization’s data set. However, anyone undertaking an unstructured data management project should be forewarned—this methodology can create a longer-term detrimental effect known as “insider knowledge.”

TechStrong Con 2023Sponsorships Available

Insider knowledge is what is left when a group of people spend time setting up data management systems within an organization and then move on from their roles without passing along critical information about the data management system to those who take over. It makes things incredibly difficult for those who inherit the agglomerated system and, chances are, the organizations who currently have this problem don’t yet realize they have it.

The Beginning of the End: Creating Insider Knowledge

Today’s storage administrators have a lot on their plate and are perpetually spread thin. Their daily responsibilities typically include:

  • Creating shares and exports
  • Designing a file system layout
  • Assigning devices for each app or dataset
  • Hunting down unused data
  • Organizing and pairing workflows to datasets

Often, with their busy day-to-day environment, the IT team’s decisions and actions are being made with a siloed approach. And as the project progresses, a unique language known only to the IT professionals overseeing the project is created. They may create their own tags and place files by a certain organizational system. However, most of this is not written down, meaning it is left to be deciphered like a code without an answer key. This language is the “insider knowledge.”

Insider Knowledge and the Great Resignation

Why is this an issue? As long as someone knows what is going on in the overall picture of the data, it’s fine, right? Well …

Solely depending on the team of IT professionals who created the workflow means that they must stick with the company forever for their systems to make sense. After all, once the people with that knowledge leave, it is largely lost forever to the company.

It is not a bet worth taking that the employees of a company will merely stick around to see the data management project to completion. The threat that insider knowledge brings to organizations is that only the people with an idea of how the system works can properly onboard the next hire. This is a great position to be in for the IT professionals who know everything about the system as there is a sense of job security, but it’s not great for the organization.

For organizations, 44% of employees in the workforce are active job seekers. The Great Resignation has showcased that job seekers have the upper hand in the market today, and that good talent can be bought and poached from companies without a moment’s notice. When—not if—an organization’s IT team members move on, it is imperative to have a well-documented, organized foundation for the unstructured data management system so that the next team is able to pick up where they left off.

Otherwise, when team members leave, they will inevitably leave knowledge gaps and questions will arise such as:

  • Are you hosting 20 TB of dead application data on your NAS?
  • Is this 20 TB of data something the storage administrator was planning on deleting, or is it important?
  • Are you going to pay to keep that data on Tier 1 disks?
  • Do you have old employee files that were archived to other storage devices but were never deleted?

The implications of insider knowledge are important to remember before beginning to tackle a data management project. It truly is as simple as future-casting and deciding not to give a few people control over an entire enterprise’s unstructured data stores.

Leave it to the Technology: What to Ask When Finding a Solution

To manage unstructured data, there are now many tools on the market built with insider knowledge (and the pre-existing codes within organizations) in mind. By automating the process as much as possible, organizations can avoid a headache when their IT team moves on. Some questions to ask to aid in finding the proper tool include:

  • Location of data: How many storage devices are there? How many data centers are there? And how many countries are the data centers spread across? Do any live in the cloud? 
  • Scanning capabilities: Do the aforementioned locations of the data affect the tool’s ability to scan it? What functions can be performed, from just scanning to analyzing with big data abilities?
  • Presentation: How do you receive the information from the tool? Is it digestible and usable? Will it easily highlight the forgotten data sets? 
  • Aftermath: When you have the information at your disposal and have the insider knowledge in front of you, how can it help you move forward to promote a healthier unstructured data environment? 
  • Time: How long will it take? How much data can the engine scan? And will this affect business-as-usual?

These are inquiries IT leaders must find answers to before deciding on the best solution for their organization’s long-term success. Regardless of how understaffed and overworked IT teams may be, in today’s modern data center, the last thing that should be done is give a few IT professionals complete control over an enterprise’s unstructured data management system.

Staff turnover is never fun. In IT, it can severely impact an entire organization if IT teams are forced to start from scratch on projects because no one is left to translate the system’s language that was built on insider’s knowledge. 

With the amount of data created each day, the time to clean up and automate the process as much as possible is now, as the time to decide on a solution that works for today was yesterday. IT leaders who take time to evaluate the new technologies and comprehensive data management solutions available will be poised to spend their budgets wisely to reduce the risk of setbacks, delays, cost overruns and even IT team turnover.

Related Posts
  • Unstructured Data Management: Avoiding Insider Knowledge Gaps
  • DevOpsQA NJ Meetup – ELK Stack
  • Panzura Launches Freedom Partner Program to Enable Solution Providers to Capitalize on Massive Shift
    Related Categories
  • Blogs
  • Business of DevOps
  • DevOps Practice
    Related Topics
  • data management
  • Great Resignation
  • institutional knowledge
  • onboarding
Show more
Show less

Filed Under: Blogs, Business of DevOps, DevOps Practice Tagged With: data management, Great Resignation, institutional knowledge, onboarding

« Agile Sucks (Redux) | Plus: DevOps on Mars
Free Dev Tools! But What’s the Catch? »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Automating Day 2 Operations: Best Practices and Outcomes
Tuesday, February 7, 2023 - 3:00 pm EST
Shipping Applications Faster With Kubernetes: Myth or Reality?
Wednesday, February 8, 2023 - 1:00 pm EST
Why Current Approaches To "Shift-Left" Are A DevOps Antipattern
Thursday, February 9, 2023 - 1:00 pm EST

Sponsored Content

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Why a Low-Code Platform Should Have Pro-Code Capabilities

March 24, 2021 | Andrew Manby

AWS Well-Architected Framework Elevates Agility

December 17, 2020 | JT Giri

Practical Approaches to Long-Term Cloud-Native Security

December 5, 2019 | Chris Tozzi

Latest from DevOps.com

Where Does Observability Stand Today, and Where is it Going Next?
February 6, 2023 | Tomer Levy
Five Great DevOps Job Opportunities
February 6, 2023 | Mike Vizard
Azure Migration Strategy: Tools, Costs and Best Practices
February 3, 2023 | Gilad David Maayan
Blameless Integrates Incident Management Platform With Opsgenie
February 3, 2023 | Mike Vizard
OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
February 2, 2023 | Richi Jennings

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

GET THE TOP STORIES OF THE WEEK

Most Read on DevOps.com

OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
February 2, 2023 | Richi Jennings
Automation Challenges Holding DevOps Back
February 1, 2023 | Mike Vizard
Jellyfish Adds Tool to Visualize Software Development Workflows
January 31, 2023 | Mike Vizard
Cisco AppDynamics Survey Surfaces DevSecOps Challenges
January 31, 2023 | Mike Vizard
Red Hat Brings Ansible Automation to Google Cloud
February 2, 2023 | Mike Vizard
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.