Logging Configuration

Ed-Fi API v8 uses Serilog for structured logging in both the Ed-Fi API and the Configuration Service. Logs are written to the console in structured format, suitable for ingestion by log-monitoring platforms such as Splunk, CloudWatch, Datadog, or the ELK stack.

The log level for each service is controlled independently:

• Ed-Fi API: LOG_LEVEL environment variable (maps to Serilog__MinimumLevel__Default in appsettings.json)
• Configuration Service: DMS_CONFIG_LOG_LEVEL environment variable

Log Levels

Level	When used	Action required
FATAL	Application cannot start or must shut down	Investigate immediately. Check for missing required configuration or resource exhaustion. Report bugs through the Ed-Fi Community Hub
ERROR	Unexpected error that interrupts service, or an external failure (e.g., database unavailable after retries)	Investigate and resolve the external issue. Report application bugs through the Ed-Fi Community Hub
WARN	Unexpected condition the system recovered from automatically (e.g., a database retry that succeeded)	Monitor frequency. Frequent warnings may indicate an underlying issue worth investigating
INFORMATION	Normal request lifecycle events — method, path, response code, duration, trace ID	Generally no action required
DEBUG	Detailed diagnostics, including anonymized request payloads (see PII Protection)	Use for integration troubleshooting. Do not leave enabled in production

Default Configuration

Production (INFORMATION)

The recommended log level for production deployments. Logs request lifecycle events — method, path, response code, duration, and trace ID — without including request or response body content.

LOG_LEVEL=Information
DMS_CONFIG_LOG_LEVEL=Information

Both services write structured JSON to the console — one JSON object per log entry. Sample output at INFORMATION level (formatted for readability):

{
  "Timestamp": "2026-06-29T14:23:45.123Z",
  "Level": "Information",
  "MessageTemplate": "{EventName}: DMS request completed: {Method} {Path} responded {StatusCode} in {DurationMs} ms with TraceId {TraceId}",
  "RenderedMessage": "HttpRequestCompleted: DMS request completed: POST /data/ed-fi/schools responded 201 in 18 ms with TraceId 0HN...",
  "Properties": {
    "Application": "EdFi.DataManagementService",
    "EventName": "HttpRequestCompleted",
    "EventId": { "Id": 1228001, "Name": "HttpRequestCompleted" },
    "SourceContext": "EdFi.DataManagementService.Frontend.AspNetCore.Infrastructure.LoggingMiddleware",
    "RequestLayer": "Frontend",
    "TraceId": "0HN...",
    "ActivityTraceId": "4bf92f3577b34da6a3ce929d0e0e4736",
    "SpanId": "00f067aa0ba902b7",
    "Method": "POST",
    "Path": "/data/ed-fi/schools",
    "StatusCode": 201,
    "DurationMs": 18,
    "PathBase": ""
  }
}

ActivityTraceId and SpanId are present only when an ASP.NET Activity is active (e.g., when OpenTelemetry tracing is enabled). Failure events use EventId 1228002 and Level Error. The file sink writes the same structured properties but omits RenderedMessage.

warning

Do not set Serilog:WriteTo:*:Args:outputTemplate in a local appsettings.Development.json override. Serilog's configuration array merging causes outputTemplate to silently replace the JSON formatter with plain text, breaking log collection. Override MinimumLevel only.

Local Development (DEBUG)

The Docker Compose .env.example ships with LOG_LEVEL=DEBUG. At this level, the Ed-Fi API logs anonymized request payloads to aid integration troubleshooting. See PII Protection for how anonymization works.

LOG_LEVEL=Debug

Production Troubleshooting

When diagnosing a difficult-to-reproduce issue in production, temporarily raise the log level to DEBUG. Return to INFORMATION as soon as the issue is captured — DEBUG logging generates significantly more output and should not be left on indefinitely.

warning

DEBUG logging includes anonymized request bodies. Although sensitive field values are replaced with "*", ensure your log storage and access policies are appropriate before enabling DEBUG in a production environment.

PII Protection at DEBUG Level

At DEBUG level, the Ed-Fi API logs anonymized HTTP request payloads. Before logging, every scalar value in the request body (strings, numbers, booleans, null) is replaced with "*", leaving only the JSON structure and property names visible. This provides enough information to diagnose request shape issues (missing fields, incorrect nesting, wrong property names) without exposing student or staff data.

Example: a POST /api/data/ed-fi/students request body is logged as:

{
  "studentUniqueId": "*",
  "birthDate": "*",
  "firstName": "*",
  "lastSurname": "*",
  "sexDescriptor": "*"
}

This masking behavior is the default (MaskRequestBodyInLogs=true). To log the full unmasked request body — for example, when debugging a parsing issue in a controlled environment — set MaskRequestBodyInLogs=false:

MASK_REQUEST_BODY_IN_LOGS=false

Correlation IDs

Every HTTP request is assigned a correlation ID that appears in all log entries for that request and is included in error response bodies. The correlation ID makes it possible to trace a specific failed request across multiple log lines and across service boundaries.

If a request does not include a correlation ID, the Ed-Fi API uses the request's trace identifier for the request's lifetime. The header name used to pass or read a correlation ID is configurable via the CORRELATION_ID_HEADER environment variable. This is disabled by default (empty string); set it explicitly to enable header-based extraction:

CORRELATION_ID_HEADER=correlationid

info

Requests that result in an HTTP 5xx response are logged at ERROR level. Requests that result in an HTTP 4xx response (including validation errors and not-found responses) are logged at Information level.

Example 1: Error response body containing a Correlation ID

{
  "detail": "Data validation failed. See 'validationErrors' for details.",
  "type": "urn:ed-fi:api:bad-request:data-validation-failed",
  "title": "Data Validation Failed",
  "status": 400,
  "correlationId": "b9567d36-91c1-4bae-aff5-3fde8367b969",
  "validationErrors": {
    "$.gradeLevels": ["gradeLevels is required."]
  },
  "errors": []
}

Example 2: Log entry for the same request

The log entry for the request in Example 1 contains the same value as TraceId, linking the structured log event to the error response body:

{
  "Timestamp": "2026-06-29T14:23:45.123Z",
  "Level": "Information",
  "MessageTemplate": "{EventName}: DMS request completed: {Method} {Path} responded {StatusCode} in {DurationMs} ms with TraceId {TraceId}",
  "RenderedMessage": "HttpRequestCompleted: DMS request completed: POST /data/ed-fi/schools responded 400 in 12 ms with TraceId b9567d36-91c1-4bae-aff5-3fde8367b969",
  "Properties": {
    "Application": "EdFi.DataManagementService",
    "EventName": "HttpRequestCompleted",
    "EventId": { "Id": 1228001, "Name": "HttpRequestCompleted" },
    "TraceId": "b9567d36-91c1-4bae-aff5-3fde8367b969",
    "Method": "POST",
    "Path": "/data/ed-fi/schools",
    "StatusCode": 400,
    "DurationMs": 12
  }
}

Specifying the Correlation ID for a Request

Client applications can supply their own correlation ID by including an HTTP header named correlationid (or the value of CORRELATION_ID_HEADER) on the request. If no such header is present, the Ed-Fi API uses the request's trace identifier for the request's lifetime.

Using a client-supplied correlation ID is particularly useful when the API sits behind a gateway or proxy that logs its own correlation IDs — matching them allows a single request to be traced end-to-end.

tip

Assign a unique correlation ID per request as early as possible in your client code. A GUID per request is a simple and reliable approach.

Example 3: Correlation ID supplied as an HTTP header

POST /api/data/ed-fi/schools HTTP/1.1
Host: localhost:8080
Content-Type: application/json
Authorization: Bearer <access_token>
correlationid: b9567d36-91c1-4bae-aff5-3fde8367b969

{ ... }

Log Collection

Both services emit the same structured request-log contract. Collector rules should target structured properties — not parse RenderedMessage.

Event IDs

Use EventId.Id to target request lifecycle events without matching on message strings:

EventId.Id	EventName	Level	Meaning
1228001	HttpRequestCompleted	Information (Ed-Fi API); Information or Debug (Configuration Service, see note)	Request completed — includes all 4xx responses
1228002	HttpRequestFailed	Error	Request failed with a 5xx or unhandled exception

note

The Configuration Service logs HttpRequestCompleted at Debug level for /.well-known/* paths (OIDC discovery endpoints). An Information-level collector will not receive these events. All other paths follow the Information rule above.

RequestLayer (Ed-Fi API only)

The Ed-Fi API emits two log events per resource API request: one from the ASP.NET frontend (RequestLayer = "Frontend") and one from the core pipeline (RequestLayer = "Core"). Endpoints that are handled directly without entering the core pipeline — such as the Discovery and Health endpoints — emit only the Frontend event. For request-count dashboards and failure-rate alerts, filter to RequestLayer = "Frontend" to count only externally visible HTTP requests and avoid double-counting.

The Configuration Service does not emit RequestLayer.

Exception Field

The Exception field is present only when the logging middleware itself observed an exception. It is omitted entirely — not set to null — when a 5xx status is produced without an observable exception (for example, when a downstream pipeline returned a 5xx status code). Collector rules must treat Exception as optional.

HTTP 413 Responses

Oversized request body rejections (HTTP 413) are emitted as HttpRequestCompleted events, not HttpRequestFailed, because they are handled as client errors. The StatusCode property carries 413.