Centralized Logging Proposal

 Centralized Logging Proposal 

 

Requirements: 

1. Low latency/ Low impact to client performance. 

2. Must be scalable, able to handle a high number of requests in parallel 

3. Quickly able to process log events. 

4. Log events must contain metadata that is useful, meaning 

     a. What application is this from? 

     b. What client is this from? 

     c. What server did this come from? 

     d. What account/ user identity is involved? 

     e. What environment is this from? 

     f. Log level (eg. Error, Log, Warning, Info, etc.) 

     g. Meaningful Record Id information (whether it be a MemberID, CaseId, or some other unique identifier, if relevant) 

5. Universally accessible and queryable interface (cross platform accessibility not dependent on language or framework-specific libraries. This lends credibility to the api model. 

6. Low maintenance/high availability (recommended that we keep this in it’s own environment) 

 

Considerations: 

1. From a scalability aspect, this logging system should be engineered such that an increase in logging events/requests will not impact performance. A messaging queue such as Hangfire should be utilized in order to mitigate this issue. 

 

2. The logging database should exist in a single location. A single SQL server with client specific databases (one for each client). 

 

3. Log records should be archived after a certain time period. We may want to utilize another database for this purpose. 

 

4. Alerting. We may want the ability to provide standardized email alerts when a threshold for an error or event occurs.