Data Ingestion with SDS v1 CSV Format

Bài viết
10/25/2023

Caution

SDS (Classic) TRANSITION CUSTOMERS: Not all fields are supported to bring data into the new SDS experience based on the v1 format. You will find them noted in SDS v1 CSV File Format under column Required? as data not supported.

Important

Also, Grade values and Course Subject values passed must align to their corresponding List of Values (ENUM) codes. If needed utilize Managing List of Values to expand Grade values and Course Subject to support code values not supported by default.

Set up SDS for Microsoft 365 Education Tenant

Microsoft 365 Education tenant
Need Global Administrator Permissions

Warning

You can use CSV (comma-separated value) files to sync your School Information System (SIS)/ Student Management System (SMS). SDS only supports uploading CSV files in the UTF-8 format and RFC compliant. Each CSV file must contain data for the fields noted 'Yes' under 'Required', the fields noted 'No' are 'Optional' data. The field headers marked as 'Required' must exist in the supplied files or they will not be accepted. The field headers marked as "Optional' are not required to be present in the files if not passing the corresponding optional data. For more information, see SDS v1 CSV file format.

Important

By connecting and making institution data available with School Data Sync, you acknowledge that you are authorized to share this data with Microsoft and commit to adhere to your organization’s data governance standards.

To access the SDS Admin Portal launch your web browser, navigate to sds.microsoft.com, and then sign in using your Microsoft 365 Global Admin account.

Select Get started.
Select Continue.

Allow a few moments for services to provision to tenant before next step.
Select Create new inbound flow and Next to continue.
Choose your data source. For CSV, you select the option Connect to my data.
Select the format. For CSV, you select the option CSV. Next to continue.
You need to provide the name of your SIS/SMS. Also, select SDS v1 CSV. Select Next to continue.
Select Browse files to present the browse and selection dialog.

Important

After your initial upload during the creation of your inbound flow, you'll need to always supply the same set of files for subsequent uploads, regardless of data changes made in the optional files or not. If you do not the system will determine that the records are no longer active and will mark them as such.
Navigate to the file location and select the files.

Refer to SDS v1 CSV file format.

Required: The three files listed below must be selected before you can progress. If you're only managing users and not classes, you only need to supply these files.
- school.csv
- student.csv
- teacher.csv
Optional / Required: Not required if you're only managing users. If you're also managing classes and class enrollments, you need to provide the three files listed below, along with the three files above. If you're not, don't select any files with these names. If one is selected and the others aren't, when you select Next, a message appears stating that one of these files is missing. You need to correct before continuing.
- section.csv
- studentenrollment.csv
- teacherroster.csv
Optional: These aren't required to proceed but may enhance experiences. If one of the files below is selected the other must also be selected to proceed.
- user.csv
- guardianrelationship.csv (Requires user and student record to build contact association.)
When you have completed selecting your files, select Next to continue. The system will validate the names of the selected files before continuing to the next screen. If a required file isn't found a message appears, and you need to address and reselect the file or files.
After the files names have been validated, the files will be uploaded to validate the structure of the selected files and are formatted correctly with the header names based on the SDS CSV v1 specification.
- If you receive the message: One or more of the files found below don't match the expected format, review the screen to see if any files aren't formatted correctly. You'll need to navigate back to the prior screen to reselect the files after correcting the issues.
- If you receive the message: We aren't able to find one or more of the optional files, review the screen to see which file or files aren't found. If you aren't expecting this message, you need to navigate back to the prior screen to reselect the files. Otherwise, select Next to continue.
After the file schema has been validated, SDS will perform a quick data validation from the files uploaded. The uploaded data pass through some of the SDS data matching and validation rules to help identify the good required and provided optional data. It isn't all the validation rules but the ones most encountered.

Tip

Data validation results are non-blocking during the onboarding or csv upload experiences, you have two options: either exit and return later after fixing the data or proceed to the next step to continue setting up your data connection and later supplying the corrected data with subsequent uploads.
For more information on the data that was flagged for investigation, select view details for the file name from the list to drill into specific items.

You'll also be able to download a report by file to get a head start on the investigation and fixes in the source system.

Flagged file columns are displayed with ability to expand and investigate what rule or rules flagged them.

By clicking on the rule, you'll be able to see another summary of the top 10 records, for example purposes.

When your investigation is complete, close the dialogs then select Next.
Choose your current academic year and provide a friendly name, such as 2023 or 2022 – 2023. (Characters must be utf8 or will be automatically stripped out).

Important

Academic year is used to associate incoming data to help build year over year historical data. The approach is aligned to an academic year versus calendar year. If your academic year spans across a calendar year, for example, start 8/15/2022 and ends 6/15/2023, the academic year value to be selected is the ending year, 2023. For more information, see Academic Year handling

Enter the dates for when your academic year starts and ends.

Enter the date when SDS should stop syncing data based on the defined academic year. It's most common to stop syncing data on the academic year end date you just entered.

Select Next to continue.
Select the user identity rule options. Make your selections for both staff and student roles.

Note

User matching is performed with the inbound flow and does not write or update the user objects in Microsoft Entra ID. The matching is performed and stored in the Education data lake. For more information, see Microsoft 365 Manage Users on the writing of the match link forward with the outbound flow.
- Attribute from source: user attribute based on data that is coming from your SIS / SMS.
  - Available source options are Username, Email, and ActiveDirectory MatchId.
- Attribute to match to: user property in Microsoft Entra ID to match to.
  - Available Microsoft Entra ID match options are UserPrincipalName and Mail.
- (OPTIONAL) If your user data doesn't include the @domain value, select a domain from the list.
Warning

Selecting a domain is optional and should only be used if the incoming data based on the selected Attribute from source does not include the @domain value. SDS doesn't check and append if the value is missing on a record. SDS will append the domain selection to all records which could result in @domain@domain and not find matches in those instances for existing user mapping with users in Microsoft Entra ID.

Caution

If the SIS / SMS users, for example users in the staff role group, could be associated to @domain1 or @domain2 or @domain3 you must have a @domain included in the source data, based on the selected Attribute from source selection (example: prefix@domain). This is needed for your Attribute to match to: UserPrincipalName or Mail to find the correct Microsoft Entra user to match with.

In the instance that a user has multiple roles, the following rules are used to determine what staff or student match rules should be used between the user record and the Microsoft Entra user object.
- If isPrimary is set for all student roles, even if association to a staff role exists, the match is made based on the student role.
- If isPrimary is set for any staff role, even if association to a student role exists, the match is made based on the staff role.
- If isPrimary is set for both staff and student role, the match is made based on the staff role.
- If isPrimary isn't set for any roles, especially with a mix for both staff and student roles, the match is based on the staff role.
Important

When configuring Microsoft 365 Manage Users flow, these rules are used if the option is activated to Create unmatched users, to define the construct for the Microsoft Entra UserPrincipalName property.

After you're satisfied with your selections, select Next to continue.
Review information presented on Review and create. If everything looks correct, select Create button. If not, you are able to navigate back to correct.
Your inbound data flow has been created and the first run is pending, select Done button. Check back to see the status of your first run.

Tip

You can set up your Manage data configuration immediately after defining your Connect data configuration, during the active first run, or later after the first run has finished.
To check on the status of the run, navigate back to the Home dashboard page.

If there are no issues with your data, the Home dashboard will state, "No Data Errors or Warnings Found" and "We did not encounter any data errors or warnings during your last run. Keep up the great work!"

If there are issues found with your data, the Home dashboard informs that ‘We found some issues with your data’ and encourage you to Investigate sync health.

Note

To manually upload changes after your first run to update changes from your source data see Update Source Data with SDS CSV

The maximum uploads, with a corresponding run, are six times per calendar day. After six times a day if you upload, it will only be run up to two more times based on SDS automated run cadence of every 12 hours.

Data Ingestion with SDS v1 CSV Format

Set up SDS for Microsoft 365 Education Tenant

SDS Access and First Time sign-in

Tài nguyên bổ sung