Blog

By Marvin Marin and Loilette Loderick

The Apparent Problem: Labor Shortage

Recent reports have indicated there is a lack of people in the IT workforce; specifically, cybersecurity professionals. The 2015 Global Cybersecurity Status Report by ISACA (Information Systems Audit and Control Association) reported an alarming 86% of business and IT professionals globally believe there is a shortage of cybersecurity professionals and a whopping 46% expected a cyber-attack on their organization in 2015 – but only 34% felt prepared. Of the more than 1,000 military and 727 civilian IT executives polled within the Federal Government, 60% said they do not have enough cybersecurity personnel to meet the demands of their mission. The 2016 Global Cybersecurity Snapshot, also by ISACA, reported that 48% of organizations surveyed planned to hire more cybersecurity professionals.  45% felt it would be difficult to find skilled candidates. In short, it’s a lot of work to keep IT networks operational and secure, and it appears there just aren’t enough people to do it, but the research masks a larger problem.  There aren’t enough people to do the work, and even those who hold certifications to do cybersecurity work may not be fully prepared to contribute meaningfully.  As cybersecurity threats and actors increase, certifications and real world experience are critical. The apparent shortage in personnel and concerns over the value of classroom certification have prompted the Department of Defense (DoD) to reconsider how potential hires are vetted. The new directive, DoD 8140, emphasizes job skills and experience over certifications alone – and replaces DoD Directive 8570, which has defined baseline information assurance training and certification for more than 10 years. DoD 8140 was based on the NIST’s National Initiative for Cybersecurity Education (NICE) standard, which breaks down security into seven categories of effort (Figure 1).

This level of categorization makes it easier to define cyber roles so there’s a common understanding of skills and capabilities across the enterprise.

The Real Problem: Skills Shortage

Lacking the personnel – the number of cyber professionals – hurts any organization or business, but in the long-term, it can be more damaging to have the position filled with a person not skilled to do the job properly. If the role is filled by an individual who holds a certification but doesn’t understand the ramifications of decisions or actions, the organization may have a false sense of security about its cyber team, which may be exploited by bad actors. On the surface the system may seem secure, since an analyst with a certification is manning the position, but more experienced or more knowledgeable hackers can be at work in very subtle ways, stealing data. In this case it would be better to have automation supporting fewer but highly skilled analysts until an appropriately skilled replacement could be sourced. Certifications are not always an accurate representation of an individual’s expertise – passing a certification exam is important but not fully indicative of the individual’s knowledge and ability to make the best decisions when confronted with real world issues.  Most often, real world experience is truly the best teacher. Some in the military community agree and have expressed concerns about the use of industry credentials as the basic measure of skill and ability. In an article for GovTechWorks (June 22, 2016) Jimmy Clevenger, director of system security engineering for the Marine Corps systems command said “You can sometimes get individuals who have multiple certifications, but that’s all they have – they cannot execute. Anyone can read a book and pass a test on it.” Even leaders in the certification community acknowledge certs don’t always reflect real-world skills. Dr. James Stanger was quoted in the same GovTechWorks article as saying “You can’t just lecture to people, throw them in a certification situation and hope for the best. The only way you can really understand how to secure a wireless network is through hands-on activity. You do it by actual practice.”  Clearly, certifications matter but practice and experience are vital as well.

Evaluating cybersecurity competency

Certifications can educate and arm an individual with the lexicon of terms, processes, and basic knowledge for the given profession, but despite the amount of time, effort, and money required to obtain many of those certifications, having a certification is just not the same as real-world experience – putting everything from the classroom and textbooks into practice in a live environment. Given that, there are things that certifying bodies need to do to improve their certification programs, better qualify and prepare the workforce, and remain relevant in a marketplace dominated by 8140:

• For practitioners at the junior to senior level:
  1. Split the certification of a candidate into two required parts – a knowledge exam and a practical component. Successful completion of the practical component demonstrates that the candidate can perform the tasks that they were tested on in a simulated environment to solve a problem (e.g., hack into a server using Metasploit)
  2. Examples of certifications that currently do this are the Offensive Security Certified Professional (OSCP) and Licensed Penetration Tester (L|PT)
• For managers and leaders that are non-practitioners who manage a security program:
  1. Certification of a candidate should be split into three parts – a review of the candidates’ resume, a knowledge exam, and peer review encompassing a defense of the candidate’s knowledge
  2. An example of such a scheme would be the Chartered IT Professional (CITP)

SANS has introduced an interesting concept of assessing a job candidate’s technical skill on behalf of an employer.  The product, CyberTalent Assessments, provides a third party assessment of specific technical skills requested by an employer and allows for that employer to see how the candidate scores against other candidates. It’s a novel concept that can act as a go-between until certifications reach the next step of maturity that we believe 8140 requires.

The Solution

IT organizations need to be cognizant of the actual problems they are facing – and perform the analysis to determine how to identify the people who are best suited for important cybersecurity roles based on certifications, real world experience, and the judgment of their peers. Equally important, the final assessment should be transferable between different federal agencies – a major goal of 8140. Another finding from the 2016 ISACA Cybersecurity Snapshot notes that 81% of the organizations surveyed would be more likely to hire a cybersecurity professional who holds a performance-based certification. Federal agencies and private companies are becoming aware of the distinction between classroom training and real world experience, and on the whole far prefer the combination of the two. Based on these evolving needs, certification bodies must re-evaluate how they certify candidates in light of new 8140 regulations.

Conclusion

The ISACA Cybersecurity findings should create concern for any organization with data or systems to protect from cyber-crimes or insider threat.  Two important questions remain:  first, can federal agencies and private employers find enough people to staff the cybersecurity positions they have available, and second, are the individuals they hire to fill those important roles capable and experienced to conduct the work? In the past, individual certifications were accepted, but increasingly we can see that classroom learning and test completion does not prove than an individual can work effectively in a complex, rapidly evolving environment. As threats increase, how we train and vet cybersecurity analysts must change. Certifications will matter, but we believe more training and practice in live, virtual environments, more coaching and mentorship, and more peer review is critical to identifying the best cybersecurity analysts.