Upload organizational data (subsequent uploads)

This article presents the steps that administrators take to upload (import) organizational data to Workplace Analytics. Complete these steps after preparing data as described in Prepare organizational data.

Important

Follow the steps in this section if this is not the first time that you have uploaded organizational data to Workplace Analytics. If this is the first time, follow the steps in Upload organizational data (first upload).

Import tasks

The task of importing organizational data has three parts:

  1. File upload
  2. Field mapping
  3. Data validation

After you prepare the source data, you can upload the .csv file and map fields. After you map fields, Workplace Analytics validates the data. When the data successfully validates, the overall data-import task is complete. If the data validation is not successful, you can choose from a few options that are described in Validation fails.

Video: Upload organizational data

File upload

In the following steps, you specify a .csv file to upload to Workplace Analytics.

To select the file to upload

  1. Open Workplace Analytics. If prompted, enter your organizational credentials.
  2. In the left navigation pane, select Settings.
  3. Select Organizational data. The Upload history area of this page displays the previous data uploads from your organization.
  4. Select New upload.
  5. On the Upload page, select Name your upload, and then type the name of your new upload file.
  6. Optionally, select Add an optional description and type a description of this upload.
  7. In the Select file section, click Select file. In the dialog box that appears, select the .csv file that you want to import.

Important upload considerations

  • The .csv file that you upload must be UTF-8 encoded.
  • Make sure that the file you are uploading is not open in a different program when you begin the upload process.
  • After the upload process begins, the process is irreversible.

Note

If you are uploading new data, go to step 8, Complete new file upload. However, if you have uploaded data and then discovered that it contains sensitive, incorrect, or unauthorized data, you must remove the uploaded data and replace it with a new file. To do this, go to step 9, Append or replace organizational data.

  1. To complete a new-file upload, select Next. This displays the System fields table. Go to Field mapping.

  2. To append or replace organizational data, locate the Append or replace area. In this area, you can select either of the following options:

    • Append the existing organization data to update attribute values for existing employees, to add new employees, or to add new attributes.
    • Replace all existing organizational data with this file to delete all previous HR data uploads and make this the first new HR data upload. If you choose this option, please note the following:
      • Permanent deletion: The replace option permanently deletes all previously uploaded organizational data.
      • Column omission: The schema of the new data file need not exactly match the schema of the previously uploaded HR data. However, omitting columns that were present in previous uploads can cause errors in auto-refresh queries that depend on the presence of those HR columns. For more information, see Prepare organizational data.
  3. After reviewing the warning message, select Continue and then map your fields.

Field Mapping

You need to map the fields (columns) for the source .csv file to the field names that Workplace Analytics recognizes. You map these on the Upload page.

Upload page

The Upload page includes tables for System fields and Custom fields for mapping the data for the upload file.

When appending new attributes to an existing upload, you need to select all the same required and optional attributes that you mapped before in previous uploads, in addition to the new attributes you want to add (append).

System fields table

System fields represent attributes that are known by Workplace Analytics and are used in specific calculations beyond grouping and filtering. A system field can be either required or optional.

  • Required fields are identified in two ways. Their rows have dark shading and show as "Required" under the Source column header. These rows represent data that was found in the uploaded file. For the upload to succeed, you must map the required fields with a column in your .csv file that is the correct data type and meets the validity threshold.

    Important

    Every required field must have a valid, non-null value in every row. This means that, even if the names of these attributes are not present in the uploaded .csv file, other columns must be present in the .csv file that are mapped to these attributes.

  • Optional fields appear below the required fields in rows that have lighter shading. These rows are commonly encountered system fields that Workplace Analytics suggests for use. You don't need to map these fields if your organization doesn't have data for them.

Custom fields table

  • Custom fields are displayed on this page below the optional fields. Custom fields are optional attributes you can create. Select a column from your source.csv file. Name the column, select the data type, set the validity threshold, and then select the report option.

Columns in the fields tables

  • Source column corresponds to each of the fields in the uploaded file.

  • Workplace Analytics name is the name of your organization's Workplace Analytics.

  • Data type is the data type of the fields.

    Note

    If the data type is Boolean, the value for the Boolean field must be TRUE or FALSE.

  • Validity threshold sets the percentage of rows in the uploaded file that must have a valid, non-null value for the attribute. The source file might still be valid even if some rows have invalid or missing values for some columns.

    Summary of Validity threshold settings

    • Required attributes: Because PersonId and EffectiveDate are required attributes, their Validity threshold value is 100%. This value cannot be changed.

    • Fields with minimum values: The Validation threshold for the ManagerId, Organization, and LevelDesignation fields is set to 95% by default, but you can raise this value.

    • Other system fields: The Validation threshold for other system fields is set to 95% by default, but you can raise or lower this value.

    • Custom fields: See Set Validity threshold for custom fields.

  • Include in report lets you decide how to treat sensitive data in the report that will be generated about the import operation.

    Map data fields

The drop-down menu under Include in report offers the following options for each of the columns in your source data:

  • Show in report: Let the actual data value display in the report just as it was imported in the organizational data file.

  • Exclude from report: Prevent the data value from appearing in the report.

  • Hash in report de-identifies sensitive data. This option includes the data in the report that it generates about the import operation, but instead of displaying the actual value that was taken from the source file, it shows a hashed version of the value – a format that cannot be read.

To map fields

After you complete the steps in File upload, the Upload page with the System fields table will appear.

  1. Map the required fields.

    a. Determine which of the columns in your .csv file correspond to the second column in the table (Workplace Analytics name):

    System fields table

    b. Under Source column (the first column in the table), click the down arrow. This displays a list of the column names that were found in the .csv file. From the list, select the correct column name for this data.

    c. Fill in appropriate values for the other columns in the table: Workplace Analytics name, Data type, and so on. Repeat these mapping steps for the rest of the required fields and for any optional fields that you choose to map.

    Note

    For more information, see Columns in the fields tables.

  2. Map the optional and custom fields, as applicable. You only need to map the columns in your source (.csv) file that your organization considers important for analysis. For example, if "StartDate" is important and your data contains this field, map it.

    Custom fields table

    a. Under Source column (the first column in the table), select the down arrow to display the list of column names that were found in the .csv file. From the list, select the correct column name for the data. In this example, you'd select StartDate.

    b. Set values for the other columns in the table, such as the data type, the validity threshold, and the hash setting for reports.

    c. Repeat these steps for all custom fields that are important to your organization.

  3. In the Submit for validation area, select I confirm that these mappings are correct, and then select Submit. This uploads the .csv file and starts the validation process.

  4. Next step is to go to Data validation.

Data validation

After you complete the steps in Field mapping, the Upload page displays the File is being uploaded screen.

Upload in progress

Note

Each tenant can have only one upload in progress at a time. Therefore you need to complete the workflow of one data file, which means you either guide it to a successful validation or abandon it, before you begin the workflow of the next data file. The status or stage of the upload workflow is shown on the progress bar across the top of the Upload page.

Important

You must stay logged in while the file is uploading or the upload will be canceled. The upload requires this page to be open in your web browser during the upload. If you close the browser (or this browser page), the upload will fail.

Validation succeeds

If validation succeeds, the Upload page will indicate it and show the size of the upload and that the overall process is complete. After successful validation, Workplace Analytics processes your new data.

Validation succeeded

You can select Settings > Organizational data Upload > Organizational data to show the Upload history page. You can then select Successes to see the workflows that were successfully validated (and uploaded).

On this page, you have the following options:

  • Select the View (eye) icon to see a summary of the validation results.
  • Select the Mapping icon to see the mapping settings for the workflow.
  • Select the Validation (download) icon to see a list of validation warnings.

Note

Each tenant can have only one upload in progress at a time. Therefore you need to complete the workflow of one data file, which means you either guide it to a successful validation or abandon it, before you begin the workflow of the next data file. The status or stage of the upload workflow is shown on the progress bar across the top of the Upload page.

Validation fails

If data validation fails, the Data load page shows a "failed" notification. It also shows details about the validation attempt and presents you with options:

Validation failed

After a failed validation, it's best to first gain an understanding of the errors by scanning the error summary table. You can also select Download issues to examine the error log.

This information about the errors helps you decide which path to choose next — whether to fix the source data, change your mapping settings, or abandon the current upload. The following section describes these options:

Options upon failed validation

Nature of errors Recommended selection Description
Minor errors, small in number Select Edit mapping This displays the Field Mapping page, on which you can change how you map source-file fields to Workplace Analytics attributes, optionally change validation thresholds, and then re-attempt validation. You can do these things without changing and re-uploading the source file. This is best for minor errors such as having mapped the wrong column in the source file or assigned a too-high validation threshold to a particular attribute.
Major errors Select Upload file This displays the first File upload page. Consider this option in the case of major errors in the originally uploaded data. First, edit the source-data file to fix those errors and then re-attempt the upload and validation process with the corrected file.

There is also an option to select Abandon, a button on the top right of the page. Select this to cancel the current upload. You can abandon your upload for any reason, related or unrelated to errors in the upload file.

Note

  • Workplace Analytics does not modify or fill in data that is missing from HR uploads, even for EffectiveDate or TimeZone. The administrator is responsible for correcting such errors or omissions.
  • When any data row or column has an invalid value for any attribute, the entire upload will fail until the source file is fixed (or the mapping changes the validation type of the attribute in a way that makes the value valid). Lowering a threshold does not ignore or skip an invalid value.

Guidelines for correcting errors in data

This section contains help for correcting data in an uploaded source file that is causing validation errors.

When any data row or column has an invalid value for any attribute, the entire upload will fail until the source file is fixed (or the mapping changes the validation type of the attribute in a way that makes the value valid). Lowering a threshold does not ignore or skip an invalid value.

All field header or column names must:

  • Begin with a letter (not a number)
  • Only contain alphanumeric characters (letters and numbers, for example Date1)
  • Have at least one lower-case letter (Hrbp); all uppercase won’t work (HRBP)
  • Have no spaces (Date1)
  • Have no special characters (non-alphanumeric, such as @, #, %, &, *)
  • Match exactly as listed for Workplace Analytics’ Required and Reserved optional attributes, including for case sensitivity (for example PersonId and HireDate)

The field values in the data row must comply with the following formatting rules:

  • The required EffectiveDate and HireDate field values must be in the MM/DD/YYYY format
  • The required PersonId and ManagerId field values must be a valid email address (for example, gc@contoso.com).
  • The required TimeZone field values must be in a supported Windows format.
  • The required Layer field values must contain numbers only.
  • The required HourlyRate field values must contain numbers only, which Workplace Analytics assumes is in US dollars for calculations and data analysis.

Note

Workplace Analytics does not currently perform currency conversions for HourlyRate data. All calculations and data analysis in Workplace Analytics assume the data to be in US dollars.

The field values also cannot contain any of the following:

  • No accent marks (á)
  • No tildes (~)
  • No short or long dashes (-, --)
  • No commas (,)
  • No "new line" characters (\n)
  • No double (" ") or single quotes (‘ ‘)
  • Limit character length of field values in rows to a maximum of 128 KB, which is about 1024 x 128 characters

Addition of a new data column

Let's say that you've already uploaded at least 13 months of snapshot data, which contained the five required columns (PersonId, EffectiveDate, LevelDesignation, ManagerId, Organization) for all employees. Now, you want to upload one new column of data – for example, an engagement score value for each employee – and you want it to apply to all of the historical data. When you upload to append the new "EngagementScore" data column, remember to reupload all five of the minimum required fields along with the new field.

Set Validity threshold for custom fields

The threshold depends on the intended use of the custom field. If you intend to use this data in much of your analysis, consider setting it to a high percentage. You can set a lower threshold for data that applies, for example, to only a small subset of people in your organization.

Set a high value

Generally, you should set the Validity threshold to a high value. This is especially important if your analysis will focus on that field.

For example, you might include a "SupervisorIndicator" attribute. At first, you might not think that you're analyzing manager behavior and you might be tempted to omit this attribute. But the organization hierarchy is used implicitly by many Workplace Analytics analyses – for differentiating different work groups, for determining high- and low-quality meetings based on how many levels attend, and more.

Set a lower value

The goal of your analysis might be to determine sales effectiveness. Your data might include an attribute for sales attainment that only makes sense for members of your sales force, who constitute about 10% of the company. This number doesn't apply to engineers or program managers, but it is critical for high-performers in sales.