Using API Tools to Review Data in the Ed-Fi ODS/API

Overview

User Story: as an education agency staff member, I want to view the data loaded into the Ed-Fi ODS/API, so that I can compare to my local source of truth and ensure the correct information has been loaded.

Solutions: Use an API client to access the ODS/API, using key/secret credentials issued by the platform host. The ODS/API's Open API files ("swagger.json" metadata) provide documentation of available resources, and there are many robust tools that can read this file to pre-create HTTP requests for you. Bruno is currently my favorite such tool; Postman and Swagger UI are more well-known alternatives.

What about database access as an alternative? And how do these API clients help solve this problem? Read on for more on these topics.

Database Access

As a general rule, only IT staff at the hosting agency will have access to query the database ("ODS") that stores the data handled by the Ed-Fi ODS/API. Even in such situations, direct database access - while tempting - is not recommended:

• It is difficult to secure row-level access to data in compliance with FERPA's "need to know" requirement.
• Direct database query permissions allow access to significant volumes of data, which can magnify the impact if someone decides to try exporting data for inappropriate or malicious use.
• When joining data from multiple tables, it is easy to make a mistake in the join parameters and accidentally associate records that do not belong together.

API Access

Reading data out of the same Ed-Fi ODS/API application that was used for writing the data is your best bet. Although the ODS/API is built for system-to-system interactions, humans can also use an appropriately-scoped key and secret via one of many competing tools that support HTTP API requests.

tip

"Appropriately-scoped" means that the credentials are:

• connected to the appropriate education organization(s),
• configured with an appropriate claimset that (a) narrows access to required endpoints and (b) limits the actions that can be performed (CRUD), and
• (optionally) prescribed a profile that further tunes which fields are "visible" in the API response.

The most useful of these tools will be able to read Open API specification files, which contain a listing of all the different Ed-Fi Resources available in the API, along with the full definition of those resources (e.g. Students, Schools, Assessments, etc.). These tools make it easy to discover which resources are available, and they make it easy to provide your key and secret, submit a request to the API, and display the JSON-formatted result.

Examples of tools in this category include:

• Swagger UI is a web application that parses Open API files and creates "interactive documentation". Indeed, the Open API specification evolved from Swagger. It comes out of the box with the Ed-Fi ODS/API source code, but many organizations choose not to deploy this in production environments. It is also possible to use Swagger UI tools online, hosted by its maker SmartBear, and developers can run Swagger UI on their own systems. The online Swagger Editor is a quick-and-easy way to access an Open API specification file. Swagger is used in the Alliance' public demonstration environment.
• Postman is probably the most well-known tool for accessing and testing REST APIs. It can import Open API files, allowing the user to create a collection of formatted HTTP requests. The Ed-Fi Alliance has made extensive use of Postman requests in the past, including for integration testing on the Ed-Fi ODS/API and Ed-Fi ODS Admin API.
• Bruno, mentioned earlier, is a new entrant in this field. It first came out in 2022, spurred by dissatisfaction with Postman's business model. It is an open source utility that has matured to the state where it is a strong competitor to Postman for most purposes. We like Bruno better than Postman because it does not require creating an account, and we prefer the way it stores its files, which is very Git-friendly. The Ed-Fi Alliance tech team recently adopted Bruno for development of test "scripts" for use while validating data submitted for the Ed-Fi Certification process. As with the other tools above, Bruno can read OpenAPI specification files to create HTTP requests templates, so that you don't have to type them in one-by-one.
• Other tools used by the Alliance's development team include:
  • Insomnia, another competitor to Postman, can also load specification files.
  • Rest Client, an extension for Visual Studio Code and Visual Studio; super light weight and useful for simple test scenarios, but it has no Open API support.

Bruno In Action

The following instructions and screenshots demonstrate use of Bruno. Postman has a similar configuration process.

1. Install Bruno.

2. Open Bruno and Import a collection:

   1. You can upload a file you've already saved, or provide a URL to load. Example: https://api.ed-fi.org/v7.3/api/metadata/data/v3/resources/swagger.json
   2. You will be prompted to choose a location on disk for saving the .bru files it creates.

3. The import sets up variables for authentication. For example, this screenshot shows variable oauth_client_id (marker #4).

   

4. Click on the environment button in the upper right to configure a new environment. The following screenshot shows the variables you must configure.

   

5. Back in the auth screen, scroll down and click the Get Access Token button. Note that the variable oauth_refresh_url is still red because no value has been provided in the Environment settings: the ODS/API only supports the client_credentials flow, which does not use refresh tokens.

   

   tip

   In the section labeled "settings", I should have clicked on "Automatically fetch token if not found" and clicked off "Auto refresh token (with refresh URL)".

6. Finally, click over to one of the requests, click the submit button, and view the JSON results: