Balisage Paper: The Model Made Me Do It! A Cautionary Tale from a Security Control Baseline Tool Developer

Introduction

This paper relates my experience developing Baseline Tailor [BT], a tool for using the United States government's Cybersecurity Framework [NIST18] and National Institute of Standards and Technology (NIST) Special Publication (SP) 800-53 security control baselines [JTF20] [JTF20B]. Baseline Tailor makes it easier for security practitioners to use these specifications together. However, despite its name, Baseline Tailor is less effective for tailoring security control baselines.

To help understand what “baseline” and “tailoring” mean in the context of cybersecurity, this discussion begins with an overview of SP 800-53 and how organizations use it to manage information security and privacy risks. SP 800-53 defines a collection of hundreds of security controls, each of which protects the confidentiality, integrity, or availability (CIA) of a system and the information it processes or transmits. Each control has zero or more control enhancements (“sub-controls”) that add functionality to or increases the strength of its parent control. This catalog of controls and enhancements is highly detailed and comprehensive, yet implementation-agnostic. Controls are grouped into families, where each family relates to a specific topic, for example, access control or configuration management. Most of these topics correspond to a security requirement specified in the Federal Information Processing Standards Publication (FIPS) 200 [NIST06].

SP 800-53 is the foundation of a Risk Management Framework (RMF) [NIST18] [Lubell20], a system life cycle-based process for managing security and privacy risk. SP 800-53 controls play a role in much of the RMF process, but these three RMF steps are the ones most pertinent to our discussion:

A system's impact, as determined in the RMF Categorize step, may be either “low” (limited), “moderate” (serious), or “high” (catastrophic). Determining system impact relies on the system's high water mark with respect to a loss of CIA. For example, suppose a system has three categories of information: proprietary information where a compromise in confidentiality would be serious, vital information where even a temporary loss of access would be catastrophic to business operations, and information where a compromise would be no more than a nuisance. The system's impact in this case would be equal to the highest CIA impact value over all information types. Since the highest of these impact values (impact of a loss of access to the vital information) is “high”, the company's system impact would be classified as “high”. The underlying logic of the high water mark concept is that confidentiality, integrity, and availability are often interdependent, so a compromise to one is likely to affect the others [NIST06].

Given the size and complexity of the SP 800-53 catalog, the RMF Select step can be daunting. To make it easier, SP 800-53 specifies three control baselines for low, moderate, and high impact information systems as starting points for security control selection. For example, an organization selecting security controls for a low impact system might begin with the controls in the baseline for the low impact level (or more succinctly, the low baseline) and tailor them as appropriate. Tailoring is the process of modifying a baseline. Examples include identification of common controls (controls that can be inherited by multiple systems to reduce IT infrastructure complexity and save money), assigning values to organization-defined control parameters (e.g., minimum password length), adding additional controls or enhancements, and providing additional guidance. A common example of additional guidance is specification of how a control is (or should be) implemented based on knowledge gained during the RMF Implement step.

My study of SP 800-53 and experience working on a project developing implementation methods, metrics, and tools to secure advanced digital manufacturing systems spurred my idea to develop a software tool for tailoring baselines. Such a tool would be helpful for creating manufacturing-specific baselines based on SP 800-53 and also could benefit would-be baseline authors in other industry sectors. The rest of this paper describes Baseline Tailor's implementation and my interpretation of SP 800-53 from the perspective of a tool developer and markup language enthusiast.

A central theme is that SP 800-53 implies multiple conceptual models of tailoring. Supplementing these implied models are derivative structured digital data models that are not officially part of the documentary standard. Collectively, these implied and derivative models provide two distinct approaches to tailoring. One approach is control-centric, which specifies tailoring in terms of the allocation of each control and control enhancement to the SP 800-53 low, moderate, and high baselines. The other approach is baseline-centric, which specifies tailoring in terms of a pre-existing baseline. This initial baseline can be one of the three SP 800-53 baselines, or it can be a pre-existing tailoring of one of the SP 800-53 baselines. As a tool developer lacking firsthand experience tailoring baselines, I failed to grasp this distinction when developing Baseline Tailor. As a result, although successful from a Cybersecurity Framework/SP 800-53 integration standpoint, Baseline Tailor achieved only mixed success with baseline authors.

The paper proceeds as follows. The second section provides examples illustrating the multiple baseline models that SP 800-53 prose and tables imply. It also contrasts the recently-released SP 800-53 revision 5 [JTF20] with the previous revision 4 [JTF13]. The third section compares two distinct structured digital derivations of SP 800-53 implied models: the SP 800-53 database Extensible Markup Language (XML) schema and the newer Open Security Controls Assessment Language (OSCAL) [OSCAL] profile model for baselines. The fourth section describes the design of Baseline Tailor and how it was influenced by implied and derivative control-centric SP 800-53 models. The fifth section discusses how a new version of Baseline Tailor will adopt a new approach to tailoring informed by both OSCAL and the tailoring criteria used in NIST SP 800-171 [Ross], a guidance publication for protecting controlled unclassified information. The sixth section concludes the paper.

Implied Models

SP 800-53 is a documentary standard that describes structured information using prose and tables (that is, a Small Arcane Non-trivial Dataset [Lubell14]). This section uses examples to describe the different baseline models that SP 800-53 prose and tables imply. It also describes an implied model for tailoring baselines from SP 800-82 revision 2, NIST's Guide to Industrial Control Systems (ICS) Security [Stouffer]. I use SP 800-53 security control AU-3 (title: “Content of Audit Records”) as the source for many examples in this and subsequent sections. AU-3 ensures that for each system event logged, the audit record contains the event type, time the event occurred, when and where it occurred, the event's source and outcome, and the identity of anyone or anything associated with the event.

Derivative Digital Models

The latest published SP 800-53 and SP 800-53B documents are considered the normative sources for controls and baselines. However, NIST also provides a non-normative searchable online database version of the controls and baselines. The SP 800-53 database [SP800-53-DB] uses an XML schema representing the SP 800-53 revision 4 and 5 catalogs^[4]. This schema is a derivative of the SP 800-53 documents and associates controls with baselines using the control-centric approach of Table I. Figure 1 shows how AU-3 from revision 4 is represented in this derivative model. The baseline-impact element allocates AU-3 and its two enhancements to the low, moderate, and high baselines in a manner mirroring Table I, except that allocation of AU-3 to the moderate and high baselines is explicit rather than implicit.

<control>...
    <number>AU-3</number>...
    <baseline-impact>LOW</baseline-impact>
    <baseline-impact>MODERATE</baseline-impact>
    <baseline-impact>HIGH</baseline-impact>...
    <control-enhancements>
        <control-enhancement>
            <number>AU-3(1)</number>...
            <baseline-impact>MODERATE</baseline-impact>
            <baseline-impact>HIGH</baseline-impact>...
        </control-enhancement>
        <control-enhancement>
            <number>AU-3(2)</number>...
            <baseline-impact>HIGH</baseline-impact>...
        </control-enhancement>
    </control-enhancements>
</control>

NIST's Open Security Controls Assessment Language (OSCAL) project [OSCAL] [Piez] has developed additional non-normative derivative digital models for representing security control catalogs and baselines (called “profiles” in OSCAL terminology). Like the low, moderate, and high baselines defined in the SP 800-53B document, an OSCAL profile references a set of controls, rather than controls referencing profiles to which they are assigned as in the SP 800-53 database schema. Figure 2 shows AU-3 as included in an OSCAL XML representation of the moderate baseline from SP 800-53B. The with-id element specifies that AU-3 and its control enhancement AU-3(1) are both in the moderate baseline. Because control enhancement inclusions are not nested within their parents, OSCAL can represent baselines such as the privacy baseline where AU-3(3) is included but its parent control is not.

<profile xmlns="http://csrc.nist.gov/ns/oscal/1.0">
   <metadata>...</metadata>
   <import href="NIST_SP-800-53_rev5_catalog.xml">
      <include-controls>
         ...
         <with-id>au-3</with-id>
         <with-id>au-3.1</with-id>
         ...
      </include-controls>
   </import>
   ...
   <modify>...</modify>...
</profile>

Unlike the SP 800-53 database schema, which does not provide a way to represent arbitrary tailoring of controls, the OSCAL profile model can represent modifications to a control beyond assignment to a baseline using the modify element shown in Figure 2. Such tailoring can include setting parameters specified in control definitions, supplementing baseline with additional controls or control enhancements, and many other possibilities. In fact, OSCAL profiles could be defined to specify the ICS overlay low, moderate, and high baselines. For the AU-4 ICS overlay specification shown in Table V, doing so would require adding a with-id element for AU-4(1) and the necessary markup and content within the modify element to represent the ICS supplemental guidance and rationale statements.

The next two sections discuss specific types of tailoring in greater detail. The next section, which describes the current version of Baseline Tailor, provide examples of baseline modification with the addition of supplemental guidance. The following section discusses an annotation-based tailoring approach, using a controlled vocabulary, planned for the next version of Baseline Tailor.

The Baseline Tailor Experience

I developed the initial version of Baseline Tailor six years ago, prior to the existence of SP 800-53 revision 5, SP 800-53B, or OSCAL. As originally conceived, Baseline Tailor's scope was limited to tailoring the SP 800-53 revision 4 security controls. The software design's early influences included the control catalog as documented in revision 4, and the SP 800-53 database XML representation of the control catalog. The revision 4 control-centric baseline allocation summaries and XML (as shown for AU-3 in Table I and Figure 1) together reinforced in my mind the idea of a user interface centered around individual controls rather than baselines. Another early influence of Baseline Tailor's user interface was the NIST ICS overlay discussed in the second section.

I envisioned the typical user of Baseline Tailor to be someone tasked by their organization with creating a tailored baseline but without intimate knowledge of the SP 800-53 catalog or tailoring methodology. To better understand my intentions, consider the following “user story” [Cohn]. Suppose Alice provides IT support for a manufacturing firm. Alice's boss Bob tasks Alice with developing a security plan for a 3D printer the company uses to produce replacement parts for internal use. Alice has a rudimentary understanding of the cyber-risk management process described in [JTF18] but has minimal experience selecting and tailoring security controls. Therefore, she decides to tailor an existing baseline, such as one of the SP 800-53 or ICS overlay baselines. Baseline Tailor is intended for users like Alice who would benefit from partial automation of the security control selection process.

I implemented Baseline Tailor in XForms [XForms], an XML application for specifying forms for the web. A user interface coded using XForms contains a model element whose content includes a collection of data “instances” that taken together express the “model” portion of the XForms model-view controller software pattern [Krasner]. Using transformation stylesheets [XSLT] to strip out markup and content from the SP 800-53 database XML download not needed for the Baseline Tailor user interface (the “...” portions of Figure 1), I could create much of my implementation's model element. Therefore, using the SP 800-53 database XML to implement Baseline Tailor had strong appeal. Figure 3 shows the Baseline Tailor security control editor user interface after a user has selected AU-3 from the Audit and Accountability family to edit. Figure 3 bears a strong resemblance to AU-3's presentation in the ICS overlay as shown in Table IV but without the ICS supplemental guidance.

Figure 3

AU-3 prior to tailoring in Baseline Tailor security control editor UI.

One portion of the Baseline Tailor XForms model I could not adapt from the SP 800-53 database XML was the model instance for representing a tailored control. This model instance, in keeping with the control-centric design of Baseline Tailor, consists of a top-level tailoredControl element representing the current state of the security control editor. Figure 4 shows the tailoredControl instance when the security control editor appears as in Figure 3. The default element represents baseline impact prior to tailoring, and the impact element represents baseline impact after tailoring. A value of 1 means the control or control enhancement is allocated to the low baseline, which implies moderate and high as well. A value of 2 indicates allocation to the moderate and high baselines, but not to the low baseline. A value of 3 indicates allocation only to the high baseline. Since no tailoring has occurred, the impact values equal the default values. The rationale element and guidance elements contain prose added by the user to explain a change to a control's baseline allocation or add additional guidance to a control or enhancement. Both elements are empty prior to tailoring. The rationale element and guidance elements each have a Boolean valued @flag attribute, which if true indicates that a resizable text box should appear for the user to provide or edit prose text.

<tailoredControl>
  <family>AUDIT AND ACCOUNTABILITY</family>
  <rationale flag="false"/>
  <control number="AU-3">
    <title>CONTENT OF AUDIT RECORDS</title>
    <default value="1"/><impact value="1"/><guidance flag="false"/>
  </control>
  <enhancement number="1">
    <title>ADDITIONAL AUDIT INFORMATION</title>
    <default value="2"/><impact value="2"/><guidance flag="false"/>
  </enhancement>
  <enhancement number="2">
    <title>CENTRALIZED MANAGEMENT OF PLANNED AUDIT RECORD CONTENT</title>
    <default value="3"/><impact value="3"/>guidance flag="false"/>
  </enhancement>
</tailoredControl>

Returning to our user story, suppose Alice uses the high water mark method mentioned in the Introduction section to categorize the 3D printer as a low-impact system and uses the ICS overlay's low baseline as a starting point for selecting security controls. Suppose also that Bob has warned Alice that one of the firm's employees, Charlene, is untrustworthy and might possibly be using the 3D printer for unauthorized purposes. Bob requests that Alice configure the printer's logging system to generate audit records that include more-detailed printer usage information, but only for employees under suspicion. Using Baseline Tailor, Alice modifies AU-3 from the SP 800-53 revision 4 default to (1) add the ICS supplemental guidance for AU-3 (from Table IV) and (2) change AU-3(1)'s baseline impact from moderate to low, thus adding AU-3(1) to the low baseline.

Figure 5 shows the security control editor after Alice's modifications. Her actions cause “Added” to appear in the “LOW” column and text boxes to appear in which she added the ICS overlay supplemental guidance and her own rationale justifying the baseline allocation change. The “XML representation” text box on the left displays current state of the tailoredControl element. This XML captures the change in a machine-readable format.

Figure 5

AU-3 after adding supplemental guidance and altering AU-3(1)'s baseline impact.

My first major update to Baseline Tailor, implemented within a year of the initial version, added the ability to browse the Cybersecurity Framework [NIST18], a hierarchically-organized taxonomy of security requirements intended to facilitate communication among stakeholders of an organization's current or target security posture. This update integrated the dense, information-rich SP 800-53 catalog model with the Cybersecurity Framework's top-down, easy-to-navigate model [Lubell16]. The result enabled users to easily use the Cybersecurity Framework and SP 800-53 together — without having to leaf back and forth between two page-oriented documentary specifications or deal with spreadsheets. Integrating the Cybersecurity Framework with SP 800-53 turned out to be a bigger success for Baseline Tailor [Kanowitz] than the control-centric tailoring user interface.

In retrospect, multiple factors led to Baseline Tailor's representation of a tailored control being a mismatch for the needs of baseline authors. I made the unfortunate choice of favoring the control-centric implied model shown in Table I and implemented in the SP 800-53 database over the more baseline-centric implied model exemplified in Table II. Neither SP 800-53B nor OSCAL existed at the time, so I was left to my own devices to create a tailoredControl model instance. The SP 800-53 database XML was an attractive option for me because XForms implementation is easiest when the underlying data already exists in XML. However, the SP 800-53 database XML is structured for search and traversing a database of controls and not for representing a baseline.

Baseline Tailor 2.0: Tailoring as Annotation

Since releasing Baseline Tailor to the public, users suggested that the security control editor user interface should be more baseline-centric. Specifically, users want to be able to tailor an existing baseline and save the results as a single XML (or JSON) document, rather than having to generate XML snippets from multiple tailoring actions and piece them together in an XML text editor. They also wanted to be able to export a partially or fully tailored baseline to a file and import it later. For a while, I was stuck on how best to revise the “tailoring” part of Baseline Tailor's XForms model. I then had an epiphany that shifted my way of thinking about tailoring away from allocation and instead toward annotation. Three things contributed to my epiphany. The first, of course, was the OSCAL baseline-centric profile model. OSCAL is an appealing format for importing or exporting baselines, given its ability to represent tailoring. Also, since the low, moderate, and high baselines from SP 800-53 revision 4 and SP 800-53B are already available in OSCAL format, I could easily give users a choice of these six baselines as a starting point for tailoring.

The second contributor was SP 800-171 (Protecting Controlled Unclassified Information in Nonfederal Systems and Organizations) [Ross]. SP 800-171, unlike SP 800-53, is targeted to a specific user community: nonfederal organizations that receive United States government funding. And SP 800-171's scope is limited to controlled unclassified information (CUI). CUI includes PII as well as proprietary and other sensitive (but not classified) information. SP 800-171 has gotten more attention recently resulting from increased awareness of the need to protect information pertaining to critical infrastructure coupled with a recent uptick in critical infrastructure cyber-attacks [Slonka].

SP 800-171 consists of a set of 110 security requirements for protecting CUI, each of which maps to one or more of a set of SP 800-53 revision 4 controls. This set of controls is specified using an annotated tailoring of the SP 800-53 revision 4 moderate baseline, defined in the SP 800-171 document as tables for each control family. Every control and enhancement in the moderate baseline has one of four annotation symbols indicating whether it:

My implementation of Baseline Tailor 2.0, currently a work in progress, includes a user interface for defining tailoring actions to be applied to a baseline as a set of ordered pairs of tailoring symbols and corresponding criteria. Figure 6 shows this user interface populated with the four SP 800-171 tailoring symbols and corresponding criteria. Figure 7 shows the model instance data representing the user-entered tailoring actions in Figure 6. Note that this user interface is sufficiently general enough to define any set of ordered pairs for any annotation-based tailoring scenario, not just SP 800-171. For example, it could be used to define an arbitrary-length set of tailoring actions for an organization-specific tailoring of the revision 5 low baseline.

Figure 6

New tailoring approach applied to revision 4 moderate baseline as in SP 800-171.

<action>
  <symbol>NCO</symbol>
  <criteria>Not directly related to protecting the confidentiality of CUI.</criteria>
</action>
<action>
  <symbol>FED</symbol>
  <criteria>Uniquely federal, primarily the responsibility of the federal government.
</criteria>
</action>
<action>
  <symbol>NFO</symbol>
  <criteria>Expected to be routinely satisfied by nonfederal organizations without 
specification.</criteria>
</action>
<action>
  <symbol>CUI</symbol>
  <criteria>The CUI basic or derived security requirement is reflected in and is 
traceable to the security control, control enhancement, or specific elements of the 
control/enhancement.</criteria>
</action>

Figure 8 lists each AU control and enhancement in the revision 4 moderate baseline and specifies via SP 800-171 tailoring action symbols which are traceable to the 110 CUI protection requirements and which are not. For example, AU-3 and AU-3(1) both have the “CUI” tailoring symbol and therefore are traceable to a requirement. AU-1 has the “NFO” tailoring symbol, indicating it is expected to be implemented (satisfied) even though no SP 800-171 requirement maps to it. AU-4, AU-6(1), AU-7(1), and AU-11 are not directly related to protecting CUI confidentiality and no SP 800-171 requirements map to them. The Baseline Tailor 2.0 user interface, in addition to enabling users to add supplemental guidance and rationale statements as does the current user interface shown in Figure 5, will also allow users to associate an annotation symbol from the set of tailoring actions they define. In this way, the user interface will be able to generate a structured digital representation of information that a baseline developer would normally have to record manually in a table within a document, or in a spreadsheet.

Figure 8

AU family moderate baseline tailoring (SP 800-171, Table E-3)

The third contributor to my shift in thinking about tailoring was a realization that an annotation-based approach obviates the need for Baseline Tailor 2.0 to explicitly support removal of a control or control enhancement from a baseline. The current Baseline Tailor user interface provides drop-down widgets in the “BASELINE IMPACT” for additions and removals, as shown in Figure 3 and Figure 5. However, as SP 800-171 shows, the annotation symbols “NCO”, “FED”, and “NFO” can be interpreted as removal plus rationale. Thus, a Baseline 2.0 user will be able to “remove” a control from a baseline simply by assigning it an appropriate tailoring annotation from a user-defined set of tailoring actions.

Adding a control to a baseline will still require more than assigning an annotation symbol. The user will need to select a control or enhancement using “Control family” and “Control” drop-downs (as in the top part of Figure 3), with choices filtered to include only those controls/enhancements in the control catalog that are not already included in the baseline being tailored. The user would then use a self-authored tailoring action to provide a rationale for adding the control/enhancement. For example, returning once again to our user story, with Baseline Tailor 2.0 Alice could define a tailoring action with symbol “THREAT” and criteria “Added to the baseline to protect against a threat specific to the system or organization.” After adding AU-3(1) to the low baseline, Baseline Tailor would prompt her to assign an annotation symbol. Alice would choose “THREAT” from the drop-down list and could add her rationale statement (“Additional audit information is needed for users deemed a security risk.”) as added supplemental guidance to further justify the tailoring action.

Conclusion

This paper discusses my multi-year education in the nuances of SP 800-53 security control baselines. While developing Baseline Tailor I discovered that SP 800-53 has more than one implied model for tailoring, and that multiple derivative digital models exist outside the normative standard. Subsequently, my mental model of a baseline evolved from a collection of individual controls with assigned impacts to an annotation of a pre-existing baseline. The upcoming version of Baseline Tailor will benefit from the lessons I learned during this journey.

No single baseline model is the correct one. The right model for a tool developer depends on the type of tool being implemented. A control-centric model such as the SP 800-53 database schema may be optimal for someone developing a user interface for browsing the catalog and its published baselines. A baseline-centric model makes the most sense for a tool that consumes and/or produces a baseline. Computer-readable digital models are more developer-friendly when the model's data format is a good fit for the implementation language they are using. Models defined as prose in standards establish a ground truth from which implementation-friendly models can be derived.

Like other successful standards, SP 800-53 exists as part of a larger ecosystem of stakeholders. The SP 800-53 ecosystem includes developers of SP 800-53, other standards supporting the RMF process, communities of interest who develop overlays and other discipline-specific guidance, policymakers who advocate for and may mandate the use of these standards, tool developers, and last but not least, those responsible for performing the RMF steps necessary to secure information systems in a risk-informed manner. As a tool developer, I found it essential to be aware of the concerns of all these stakeholder groups. I also learned not to let my predisposition for a single model or implementation technology distract me from considering alternatives that might be a better fit for my tool's purpose.