Blog

Part 1 of a 2 part series on Data Consolidation in the Federal Government

By Colby Proffitt

Doing more with less – that’s been a common theme in the Government IT space, even more so in the last decade. Budget growth has slowed dramatically while information technology capabilities and user demands skyrocket. Finding ways to create efficiencies and provide new solutions for customers with flat budgets is paramount. Data Center rationalization, transformation, and consolidation offers perhaps the best opportunity to provide the same or better service with a smaller footprint and greater efficiency. Government agencies think so too: in August 2016, the Office of Management and Budget (OMB) finalized a new data center consolidation directive seeking 25% cost reductions through 2018. The directive encourages data center consolidation and notes that any recommended new data center construction budgets won’t be approved without prior consent of OMB. While data center consolidation and rationalization promise great benefits, these consolidations present significant challenges.

In Part I of this series we’ll highlight some of the challenges that data center consolidation presents – those facing CIOs and decision makers in big organizations, as well as those facing the end-users. In part II of this series, we’ll provide recommendations on how to overcome those obstacles.

Data center consolidation offers the promise of scalability, efficiency, lower management costs, and enhanced cybersecurity. While these benefits are achievable, it helps to understand the work required to realize those benefits, and the challenges that data center consolidation presents. We believe there are at least five significant challenges to consider:

• Planning and communicating effectively
• Consolidating not only data centers but also roadmaps and priorities
• Considering leadership changes and needs for new types of leadership
• Adjusting to changes in working processes and tools
• Ensuring data availability, reliability, and security

Most data center consolidations begin by focusing on the technical challenges, which are significant, but many of the consolidation challenges have more to do with changes in processes and expectations in new or merged personnel roles, rather than changes in technology or tools. We’ll examine each of these data center consolidation challenges and present examples based on our experience to help illuminate the decisions during a consolidation and the potential implications of decisions taken during the consolidation.

1.)   Realizing, planning, scoping, and socializing the consolidation

Pressured by flat budgets, growing demands on information technology, and directives mandating change, senior executives and decision makers are required to consolidate networks, systems, and data centers; however, they may not be aware of the technical implications, or they may be so far removed from the applications and tools that they may not realize the impact to those who do use those tools. Changes in technology reach beyond the obvious impact on IT and extend to the ways people use and manage systems and information to accomplish their job. There is also an impact on the way they interact, communicate, and share data with each other. It’s easy to calculate the financial impact in terms of savings, but it’s also important to identify the impact – positive or negative – on the organization, its mission, and its people.

It’s imperative to plan properly and to set expectations for large consolidation efforts. Proper up-front planning will save organizations significant funds and reduce roadblocks down the road. Customer frustration is inevitable during consolidations, so it is important to create a comprehensive socialization plan prior to the start of consolidation efforts. By performing due diligence in this phase and locking down the scope and requirements of the consolidation, you will minimize the potential for scope creep during the migration, which can have catastrophic impacts to large scale consolidations.

Consolidation can rationalize, simplify, and transform information technology networks, systems, and data centers that have grown in a piecemeal fashion, often in response to urgent demands but without careful strategy. However, consolidation doesn’t simply reduce the number or type of computer systems or data centers; consolidation also combines organizational cultures, management styles and prerogatives, IT philosophies, and organizational history. These less tangible issues are often a significant challenge to a data center consolidation, and frequently the impact of these issues is overlooked.

For example, one of our customer organizations was divided into separate sub-organizations, with each independently responsible for its own IT – each had its own service desk, IT staff, funding, and support model. When service degradation occurred, users had to rely on siloed IT teams with limited resources to identify and resolve the issue – and there was rarely any way of easily tracking how expansive the problem was (i.e., a single user versus multiple sub-organizations). As part of the consolidation, a single service desk was created to serve all users across the enterprise. With an expansive knowledge base, 24/7 support staff, and a significantly more holistic view of the entire network, not only was the overall footprint reduced, but the level of support was markedly improved, ultimately improving customer satisfaction. An additional benefit to this consolidation was that the IT staff, from Tier 1 support to the most senior IT staff, quickly became better versed with a wider set of technologies and were better able to discover when systemic issues were occurring.

Consolidation can only be successful, however, with proper preparation and training of staff to handle the wider range of support issues. Proper preparation can also help reduce customer frustration when dealing with a new support team that has different escalation policies or staffing. Proper education and preparation, as well as expectation setting will remove the fear of poor or inadequate service from a support team, building trust and demonstrating reliability quickly.

2.)   Consolidating roadmaps and priorities

Depending on the size of the organization being consolidated – the number of networks, systems, data centers, and most importantly, people being consolidated as well as the number of people managing the consolidation – it can be difficult to keep focus on the primary mission of the organization as well as existing individual projects. With deadlines, process changes, tool changes, and the likelihood of unforeseen problems, leaders must make it a point to keep the mission the number one priority, while also sustaining existing projects; if the mission is not accomplished, then the organization or business will fail and the consolidation efforts will be meaningless. Regardless of the ease or difficulty of the consolidation, maintain the mission. It can be easy to get caught up in the day-to-day challenges of consolidation and lose sight of the larger mission, providing services and support for the customer and those who support them.

In some cases, while the technology benefits expected from consolidating two organizations may be obvious, the missions of those two organizations may pull resources and funding in different directions. For example, prior to the consolidation of all the sub-organizations mentioned earlier, system and application administration was executed independently by each component, causing complexity within the shared infrastructure and introducing risks to configuration management, cyber security, capacity management, and investment planning. In order to successfully execute data center transformation, leaders must understand the organization’s IT portfolio to determine the utilization of each investment, and to ensure that the capability still provides tangible value, while meeting the intent of the organization’s strategy.

As another example, during a consolidation effort for two other customers, challenges arose with the private cloud infrastructure. Both teams had a portion of responsibility in building out the service, but these lines and responsibilities quickly became blurred during the consolidation. Had management not quickly identified the gaps and aligned the teams appropriately, the private cloud solution would have been, at best, muddled and incomplete, and, at worst, would have failed to meet objectives. By emphasizing communication and coordinating the private cloud infrastructure activities as part of the consolidation, we were able to deliver a valuable and useful private cloud capability while consolidation efforts continued in parallel, maintaining the mission and meeting the objectives of the agency.

This concern is not limited only to technology issues, but across all facets of your organization. For example, the department that handles contracts and license renewals must now track a potentially larger number of items. Is the team adequately staffed to handle the increased work load? Do they have all the necessary information to ensure contracts and licenses don’t lapse? Do the existing processes account for license cancellation, alteration, and renewal?

3.)   Changes in leadership and staff

Data, systems, and networks aren’t the only things that get consolidated. When organizations merge or simply downsize, there are often impacts on staff and leadership alike. Where there were once two CIOs, there may now only need to be one; and where there were once two teams supporting two groups of users, there may now only be one centralized team for an entire user base. Not only does leadership have to adjust to a different, often broadened set of responsibilities, but support staff have to adjust to new direction, new responsibilities, new processes and new staffing levels that may be significantly different from their previous experience. This is an adjustment for the IT staff, and importantly, for the customers of the data center or system who may lose familiar support routines or need to learn new methods and procedures.

Again, socialization and communication of pending changes are key to making the transition successful. Existing staff must understand what they are responsible for, who they report to, and who their customers are under a new operating model. In addition, customers must understand the new method of obtaining support, and new service level expectations. This can be mitigated with up front planning and agreed upon strategies before consolidation starts.

For example, one organization may have a large Tier I support staff that can field and resolve 80% of issues as they are reported, and may only need a small staff for Tier II and a handful of engineers for Tier III. Another organization, however, may have already shifted a portion of Tier I responsibilities to a self-service/automation model, eliminating the need for a large support staff at Tier I, and requiring support staff mostly at the Tier II level. Both Tier III staff levels for each organization may be similar in size, but the two organizations will need to reevaluate their models for Tier I and II – either shifting to automation and self-service (which will also mean an adjustment for end users who are used to more personal Tier I support) or changing staffing levels at Tier II and III to match the demand of the users. There is no definitively right or wrong way to adjust – leadership must evaluate the needs of their newly consolidated organization.

Another factor that organizations may overlook is the impact of change on people. These organizations are made up of people who provide services to customers, and on the whole, people tend to dislike change. Further, as the change is unfolding, they may need more support and encouragement to accept the changes and continue to deliver excellent customer service to help support the mission. Setting expectations early, effectively communicating throughout the change, and listening to concerns can go a long way in helping people bridge the gap from the way things used to be in the old IT environment to the way they will be in the new environment.

4.)   Changes in process and tools

Similar to changes in leadership and staff, there are usually changes to the processes and tools data centers count on to deliver services. Standing alone, an organization may have the tools and processes it needs to operate effectively and efficiently, but when merged with another organization, leadership will have to evaluate the needs and requirements of those users and determine if they need to purchase a new tool, or train users on a tool they already have. Either way, users may find it difficult to switch to a new platform and leadership should expect the end users to voice concerns and allow adequate time for training and user acceptance. For example, in the federal space, many organizations use Remedy as an IT Service Management (ITSM) tool; however, some organizations are on version 7.6, some are on 8.1, and others are on version 9.0. And, even if they have the same version, they may have configured the tool differently. In some cases, consolidation may require two organizations to choose between different ITSM tools, such as Remedy and ServiceNow, which presents not only a selection challenge but an educational and training challenge for both the IT staff and the end users. When it comes time to consolidate, evaluate the requirements of your new user base and make sure you select the tools that your users need that fit within your budget and achieve your mission.

This sort of conflict needs to be understood up front and mitigated prior to the beginning of the consolidation. For example, if two versions of Remedy exist, the answer may be that all Tier I/II staff get accounts on both systems, and both run in parallel until a new consolidated version is deployed.

It is pertinent to plan for as many process changes/consolidations as possible. Most organizations rely on IT for everything from onboarding and account creation to hardware and software tracking. While most of the more frequently used processes will be taken into consideration when planning, it can be easy to overlook the processes that aren’t used as often.

When a process is overlooked, it can cause unnecessary delays in consolidation, and lead to customer confusion and frustration. For example, during a consolidation for one of our customers, although an overall accounts management process was created, one independent tool was not accounted for. When a new customer came on board with a requirement for an account for that tool, the process workflow didn’t work, as the accounts management team did not have the ability to create those accounts, and the team no longer had rights to link to Active Directory accounts. This led to account delays and took time to resolve. However, these situations can be avoided and should be accounted for by planning for daily or weekly meetings during and post migrations.

5.)   Data Integrity, Security, and Reliability

Many organizations will turn to virtualization as a means to consolidation. While virtualization on its own isn’t necessarily insecure, it is less secure than physical servers because of the increase in entry points to the network and data. The virtual network can quickly become intricate and dynamic, making it more difficult to secure. Security researchers at the National Institute of Standards and Technology (NIST) agree that virtualization can have negative implications in terms of security. According to NIST’s Guide to Security for Full Virtualization Technologies, adding layers of technology can increase the security management burden by requiring additional controls.

That’s not to say that virtualization isn’t a good option – it just requires careful planning. Organizations that plan to use virtualization to consolidate should be sure to continuously audit virtual machine hosts against approved configuration and patch management baselines to reduce the risk of an attacker exploiting a vulnerability on the host or hypervisor. The addition of management controls can help isolate and prevent unmanaged VMs from being introduced in secure environments by implementing rogue detection and isolation. By continuously monitoring and managing the VM usage, resources, and overall security posture, the organization can ensure that the host is both secure and is meeting the strategy of the organization. Ultimately, this type of approach can also result in cost savings through a significant reduction in VMs supporting legacy requirements, improving the overall security stance, while also allowing the organization to repurpose resources for new organizational requirements.

Regardless of the means to consolidation, organizations must create a detailed plan to specifically address security, both at the logical and physical level. Concerns such as data center room access and ensuring file and folder permissions are migrated properly are key, as well as tracking which administrators have access to which tools. When this planning step is overlooked, it becomes tempting for IT Staff to “open to all” during migration so people can complete their tasks, with the intent of cleaning up and locking down the systems after migration. Unfortunately, the cleanup portion is often skipped or not completed properly, leaving the consolidated system less secure than the previous independent systems.

Migration plans must also account for system uptime, data integrity, and security related items during the migration. For example, it can be easy to lose track of critical tasks such as daily backups during a migration, but backups are even more imperative to be completed reliably when undergoing a migration.

Conclusion

On the surface, consolidating two organizations with seemingly similar missions may appear to be an easy task. However, upon looking closer, leadership may find differences in staffing, procedures, policies, and a myriad of different applications, tools, and software. Dealing with these differences while sustaining the mission of the two organizations – and identifying the new mission of the newly-formed, consolidated organization – quickly becomes a much more challenging and daunting activity.

There are different challenges for everyone in the organization – from leadership and decision makers to end users, and the IT staff completing the consolidation.

To recap:

1.) Developing a sound consolidation plan and strategy and communicating it effectively to everyone in the organization is critical to expectation management, mission sustainment, and customer satisfaction.

2.) Identifying the objectives and roadmaps of the organizations being consolidated and achieving consensus on the new priorities of the consolidated organization will allow the two organizations to sustain their current missions and transition successfully into a new environment.

3.) Planning and adjusting for organizational changes – from leadership and management to processes and operations – will allow the organization to achieve the maximum benefits from the consolidation.

4.) Realizing and planning for the impact of the consolidation on the people within the two organizations and communicating and socializing the coming changes will help prepare everyone across the organization for adjustments and differences in processes, leadership, the organizational hierarchy, and the way the business operates.

5.) Identifying your data, security, and networking needs and researching the best way to consolidate them will ensure that you not only maintain your mission and IT integrity, but that you also align your data consolidation efforts with the latest government mandates and requirements.

In Part II of this series, we’ll explore more detailed recommendations on how to avoid and overcome the challenges discussed here, as well as some considerations to help prepare for and achieve a successful consolidation effort. Be sure to follow us on Twitter @NetCentricsCorp to hear about the next installment in this series on consolidation.