Instrumentation SDK (legacy agent)
The following content pertains to
AppOptics agents are no long receiving updates. The new SolarWinds Observability libraries are regularly updated with new features and improvements. If you are still relying on the AppOptics agents and your components are supported by the new libraries, consider migrating to SolarWinds Observability.
If you have already transitioned to the new SolarWinds Observability NET Library, see the SolarWinds NET Library documentation for the instrumentation SDK information.
SolarWinds Observability libraries are not compatible with AppOptics agents. Do not use a mix of SolarWinds Observability libraries and AppOptics agents to instrument applications that are part of a distributed trace.
Get started
Appoptics.Agent.Api NuGet package
Using the SDK nuget package requires the .NET Agent to be installed. To use the SDK in a .NET Framework or .NET Core application, add AppOptics.Agent.Api.nupkg
to your project.
AppOptics.Agent NuGet Package
The SDK is also part of the AppOptics.Agent NuGet package. The NuGet package can be added to your project and provides both auto-instrumentation and custom instrumentation functionality.
For applications using the NuGet package that are running on a server with the .NET agent installed, install or upgrade the NuGet package to version 4.3.0 or above.
Customize out-of-box instrumentation
Customization involves adding hooks from our public SDK to your code so that you can to take advantage of additional filtering capabilities on the dashboard, change how requests are traced, or capture additional information during the trace.
Starting a trace
Sometimes out-of-the-box instrumentation might not capture traces on certain systems (custom web framework, console application, etc.), but traces can still be created by using the Trace.StartTrace() and Trace.EndTrace() SDK methods, and the optional Trace.WaitUntilAgentReady() call to ensure agent readiness.
Also see the section tracing across external spans on how to use the Trace.ContinueTrace() method to continue a trace from an upstream service.
Custom spans
By default, the .NET Framework instrumentation creates a span for the code itself, e.g., ‘IIS', ‘WCF-Service', ‘ado-mssql' and the .NET Core SDK Middleware creates a root span for requests, ‘aspnet-core'. If these spans don't provide enough visibility, you can further divide your code into sub-spans. How you segment your application modules and sub-systems into spans is entirely up to you. For example the out-of-box instrumentation might not recognize calls to external processes, so you could use the SDK to manually create spans for them. Similarly, multi-threaded apps can have individual spans for each of its asynchronous operations. To create a custom span, identify the section of code you would like to appear as a span, and wrap it in a pair of entry and exit events as shown.
EntryTraceEvent event = Trace.CreateEntryEvent("my_span_name");
event.AddInfo("something", "interesting");
event.Report();
...
ExitTraceEvent event = Trace.CreateExitEvent("my_span_name");
event.Report();
Two ways to create a span
The attribute method of adding a custom span is only applicable to the .NET Agent SDK.
If you want to create a custom span for a particular method, there are two ways to do this. The first is to wrap it in a pair of entry and exit events. But this technique becomes burdensome if the method is called more than once and you want to trace each of those calls. Instead, you can use the second way which is the TraceAttribute attribute class. Attribute classes are a convenient way of associating declarative information with program entities, in this case the method you want to trace. The syntax for associating the TraceAttribute with a method is shown below; try to declare the association as close to the method as reasonable, for example within the class just above the method definition. TraceAttribute requires one positional parameter, which must be the method name; the method name will also become the custom span name. TraceAttribute accepts to two optional named parameters, see backtraces and return values for more on that.
class MyClass {
[TraceAttribute("AnnotatedMethodcharRef", Backtrace = true, ReturnValue = true)]
public void AnnotatedMethodcharRef(ref char value)
{
// your method definition goes here
}
}
Tracing multi-threaded applications
The following steps are only applicable for .NET Framework 2.0 - .NET Framework 4.5 applications. For .NET Framework 4.6 and higher and .NET Core the context is automatically propagated to child threads.
You may want to report events from background/child threads and associate them with the thread that spawned them. Assuming that a trace was already started in the parent thread:
- Pass the trace context to the child thread using TraceContext.GetCurrent().
- Set the context within the child thread using Trace.SetAsCurrent().
- Mark the thread as asynchronous by calling EntryTraceEvent.SetAsync() on the entry event of the child span.
- Clean up the thread by calling TraceContext.ClearCurrentContext() before returning it to the threadpool.
If the context needs to be passed to the exit event creation, TraceContext.GetCurrent() must be called after the entry event is reported to get the latest context. The latest context can then be passed to the exit event creation. SetAsCurrent() would need to be called on the passed context and then the exit event can be created.
System.Threading.Thread myThread = new System.Threading.Thread(this.DoSomeWork1);
myThread.Start(TraceContext.GetCurrent());
public void DoSomeWork1(object data)
{
TraceContext currentContext = data as TraceContext;
if(currentContext != null)
{
currentContext.SetAsCurrent();
var asyncEntryEvent = Trace.CreateEntryEvent("my_span_name");
asyncEntryEvent.AddInfo("something", "interesting");
asyncEntryEvent.SetAsync();
asyncEntryEvent.Report();
// Code that needs to be traced
var exitEvent = Trace.CreateExitEvent("my_span_name");
exitEvent.AddInfo("something", "interesting");
exitEvent.Report();
TraceContext.ClearCurrent();
}
}
Tracing across external spans
While instrumenting your code you might want to trace a request across processes or hosts. This can be done by passing the X-Trace ID between them.
- The client should call
Trace.GetCurrentTraceId()
, which returns a string representing the current trace. If the request has been sampled a trace ID will be returned regardless of if the request is being traced or not. The trace ID indicates if a request is being traced by ending in01
. - The client should then send the value to the remote process by any desired means, but probably via the X-Trace HTTP header.
- The remote process should read the identifier and input it to
Trace.ContinueTrace()
. - When the remote process is done, it should end the trace using
Trace.EndTrace()
. - The remote process should then report the end trace event using
EndTraceEvent.ReportAndReturnTraceId()
, which returns the trace ID. - The remote process should send the trace ID back to the client by any means, but probably via the X-Trace HTTP header.
- The client should then associate the trace ID with the next exit
event that it reports by using
ExitTraceEvent.AddEdge()
, which links the client-side and remote-side events.
Collecting additional trace details
Any additional information you collect will be presented on the trace details page.
Backtraces and return values
Backtraces and return values may be included when you are creating a custom span for an individual function by way of the TraceAttribute class. To include a backtrace in the entry event of the span, set the backtrace parameter of the TraceAttribute class to ‘true'. To include a return value in the exit event of the span, set ReturnValue to ‘true'.
Add info events
There are two reasons you might want to create an info event: the first is to simply to attach any meta data that may be of interest to you during later analysis of a trace. The other reason is to take full advantage of AppOptics filtering capability. In short, you can classify extents by attaching a pre-defined set of specified key/value pairs. Then from the dashboard, you can filter for traces with that extent type. See special interpretation, for more on this. To add info to an extent, after creating the info event, call .AddInfo() on it one or more times as shown. In this case the information events are displayed on the raw span data tab on the trace details page. If all of the key/value pairs for special interpretation are present, the extent type is reflected in the span details tab.
InfoTraceEvent event = Trace.CreateInfoEvent("my_span_name");
event.AddInfo("something", "interesting");
event.AddInfo("something_else", "also_interesting");
event.Report();
Report errors
Create an error event for an exception, including a backtrace.
try {
// your code that might throw an exception goes here ...
} catch(exceptionType exception) {
Trace.ReportException(exception);
// the rest of your exception handler ...
}
Complete Examples
Just want to read some code? Below are examples that use the .NET instrumentation SDK to illustrate custom instrumentation concepts such as starting a trace and adding spans for an application.
This demonstrates instrumenting a simple console application written in C#. Note that once the trace is started via SDK, any execution path through supported components will be auto-instrumented without needing extra SDK calls. The example code can be used in either a .NET Framework, .NET Core or .NET application.
using AppOptics.Instrumentation;
namespace ConsoleExampleApp
{
class Program
{
private const string CONSOLE_SPAN_NAME = "console-example-app";
static void Main(string[] args)
{
// The Trace.WaitUntilAgentReady method is used to ensure that the agent is ready before the
// console application is ran so that a trace will be generated.
// The timeout parameter is set to 2000 milliseconds, if the agent is not ready in 2000 milliseconds
// the method will stop waiting and return false.
var agentReady = Trace.WaitUntilAgentReady(2000, out int statusCode);
// Debug logging if agent ready returns false
if (!agentReady)
{
System.Console.WriteLine("Agent isn't ready. Status code returned: {0}", statusCode);
}
// Start the trace by creating a StartTraceEvent and reporting it
StartTraceEvent startTraceEvent = Trace.StartTrace(CONSOLE_SPAN_NAME);
// The optional key "TransactionName" can be added to the start trace event
// If a transaction name isn't specified, the transaction name is set to "custom_<start trace span name>"
startTraceEvent.AddInfo("TransactionName", "console_test_command");
startTraceEvent.Report();
// The transaction name can also be set anytime before the end of the trace using the Trace.SetTransactionName method
// Example: Trace.SetTransactionName("transaction_name");
// Note: Thread.Sleep is used in the console example application so that spans with different timings
// are displayed when viewing traces in AppOptics
System.Threading.Thread.Sleep(200);
ProcessCommand(args);
System.Threading.Thread.Sleep(100);
// End the trace by creating a EndTraceEvent and reporting it
// Note: Span name used for start trace event must match the span name for the end trace event
EndTraceEvent endTraceEvent = Trace.EndTrace(CONSOLE_SPAN_NAME);
endTraceEvent.Report();
}
public static void ProcessCommand(string[] args)
{
// The EntryTraceEvent and ExitTraceEvent can be used to create additional spans in the trace
// The span name of the EntryTraceEvent must match the span name of the ExitTraceEvent
EntryTraceEvent entryTraceEvent = Trace.CreateEntryEvent("command_one");
entryTraceEvent.Report();
// do command work
System.Threading.Thread.Sleep(500);
ExitTraceEvent exitTraceEvent = Trace.CreateExitEvent("command_one");
exitTraceEvent.Report();
}
}
}
This demonstrates propagating trace context between two simple console applications written in C# for .NET Core. The calling application uses StartTrace to start tracing, GetCurrentTraceId to get the current trace context and passes it to the called external application.
If the external application returns a trace context (it is instrumented), the caller adds it as an edge to the current span to indicate the point where a synchronous remote call returned.
using System;
using AppOptics.Instrumentation;
using System.Diagnostics;
using System.IO;
namespace coretestapp
{
public class Example
{
string spanName = "example-trace";
public void StartTrace()
{
System.Console.WriteLine("Starting trace");
StartTraceEvent startEvent = AppOptics.Instrumentation.Trace.StartTrace(spanName);
startEvent.Report();
}
public void EndTrace()
{
System.Console.WriteLine("Ending trace");
EndTraceEvent endEvent = AppOptics.Instrumentation.Trace.EndTrace(spanName);
endEvent.Report();
}
public void ExternalServiceSpan()
{
System.Console.WriteLine("\n****** Enter ExternalServiceSpan ***** \n");
System.Console.WriteLine("Starting span for ExternalServiceSpan");
string spanName = "example-span";
EntryTraceEvent eventEntry = AppOptics.Instrumentation.Trace.CreateEntryEvent(spanName);
eventEntry.Report();
// Launch external app and pass the current trace id
string xTraceId = AppOptics.Instrumentation.Trace.GetCurrentTraceId();
string xReturnedTraceId = LaunchExternalApp(xTraceId);
System.Console.WriteLine("Ending span");
ExitTraceEvent eventExit = AppOptics.Instrumentation.Trace.CreateExitEvent(spanName);
// Add trace returned by external application
eventExit.AddEdge(xReturnedTraceId);
eventExit.Report();
System.Console.WriteLine("\n****** Exit ExternalServiceSpan ***** \n");
}
public string LaunchExternalApp(string xTraceId)
{
string returnXTraceId = String.Empty;
// Path to external application directory
string path = Directory.GetCurrentDirectory();
#if DEBUG
path += @"\..\..\..\coretestappexternal\bin\Debug\netcoreapp2.0\";
#else
path += @"\..\..\..\coretestappexternal\bin\Release\netcoreapp2.0\";
#endif
ProcessStartInfo processStartInfo = new ProcessStartInfo();
processStartInfo.Arguments = path + "coretestappexternal.dll" + " " + xTraceId;
processStartInfo.FileName = "dotnet.exe";
processStartInfo.WindowStyle = ProcessWindowStyle.Hidden;
processStartInfo.CreateNoWindow = true;
processStartInfo.RedirectStandardOutput = true;
processStartInfo.WorkingDirectory = path;
int exitCode;
// Run the external process & wait for it to finish
try
{
Process process = Process.Start(processStartInfo);
// Read standard output from external application
string output = process.StandardOutput.ReadToEnd();
// wait for process to exit
process.WaitForExit();
// retrieve application exit code
exitCode = process.ExitCode;
// when application exit with success process std output
if(exitCode == 0)
{
var lines = output.Split("\r\n", StringSplitOptions.RemoveEmptyEntries);
foreach(var line in lines)
{
System.Console.WriteLine("OUTPUT: {0}", line);
if(line.Contains("x-trace-id"))
{
var parts = line.Split("=", StringSplitOptions.None);
if (!string.IsNullOrEmpty(parts[1]))
{
returnXTraceId = parts[1];
System.Console.WriteLine("External application returned trace {0}.", returnXTraceId);
}
}
}
}
}
catch (Exception e)
{
Console.WriteLine("Exception: {0}.", e);
}
return returnXTraceId;
}
public void Run()
{
StartTrace();
ExternalServiceSpan();
EndTrace();
}
}
public class Program
{
private const int WAIT_AGENT_TIMEOUT = 5000; // Wait timeout for agent to be ready
public static void Main(string[] args)
{
System.Console.WriteLine("Running application...");
// Wait until agent is ready
System.Console.WriteLine("Waiting until agent is ready...");
bool bIsReady = AppOptics.Instrumentation.Trace.WaitUntilAgentReady(WAIT_AGENT_TIMEOUT);
if (!bIsReady)
{
System.Console.WriteLine("Agent is not ready. Exit application.");
return;
}
System.Console.WriteLine("Agent is ready.");
// Run example
Example test = new Example();
test.Run();
System.Console.WriteLine("Application has stopped.");
}
}
}
The called external application passes the received trace context to ContinueTrace to add more spans to the trace in progress. On exit, it returns the updated trace context to the caller by writting it to standard output.
using System;
using System.Threading;
using AppOptics.Instrumentation;
namespace coretestappexternal
{
public class Program
{
private const int WAIT_AGENT_TIMEOUT = 5000; // Wait timeout for agent to be ready
private static string spanName = "example-trace-external";
public static int Main(string[] args)
{
if (args.Length != 1)
{
System.Console.WriteLine("Invalid number of arguments. Expecting x-trace-id passed as argument.");
return -1;
}
System.Console.WriteLine("Running external application...");
// Extract x-trace-id as first argument for the program
string xTraceId = args[0];
System.Console.WriteLine("External application launched with trace id {0}.", xTraceId);
// Wait until agent is ready
System.Console.WriteLine("Waiting until agent is ready...");
bool bIsReady = Trace.WaitUntilAgentReady(WAIT_AGENT_TIMEOUT);
if (!bIsReady)
{
System.Console.WriteLine("Agent is not ready. Exit external application.");
return -1;
}
System.Console.WriteLine("Agent is ready.");
// Continue trace using the provided x-trace-id
ContinueTraceEvent continueTraceEvent = Trace.ContinueTrace(spanName, xTraceId);
continueTraceEvent.Report();
// Do some work
int count = 5;
while(--count > 0)
{
Thread.Sleep(1000);
}
// End trace
EndTraceEvent endTraceEvent = Trace.EndTrace(spanName);
// Retrieve the x-trace-id to be returned to launching process
string returnXTraceId = endTraceEvent.ReportAndReturnTraceId();
// Write x-trace-id to standard output to be extracted by launching process
if (!string.IsNullOrEmpty(returnXTraceId))
{
System.Console.WriteLine("x-trace-id={0}", returnXTraceId);
}
System.Console.WriteLine("External application has stopped.");
return 0;
}
}
}
.NET SDK reference
The following methods are provided by the .NET instrumentation public SDK. The SDK namespace is: ‘AppOptics.Instrumentation'.
Trace.StartTrace()
method: |
Trace.StartTrace() |
---|---|
description: |
Create a new span if a trace is already in progress or start a trace. This would typically be done when a new request enters your system. For the .NET Framework, nine times out of ten your incoming requests are fielded by a webserver and so traces will be initiated there by our instrumentation. For the .NET Core SDK, if your application is an ASP.NET Core service and you add the AppOptics middleware to your application then traces will be initiated by our middleware. If you do need to start traces within your application code, note that the first span is started automatically, you don't need to add a entry event for it. |
returns: |
An event that can be populated with key/value pairs and reported. |
parameters: |
|
history: |
Introduced in version 1.4.0 |
example: |
|
notes: |
|
related: |
Trace.ContinueTrace()
method: |
Trace.ContinueTrace() |
---|---|
description: |
Either creates a new span if a trace is already in progress or continues a trace from an external span. This method is used almost exclusively for tracing across external spans. |
returns: |
An event that can be populated with key/value pairs and reported. This event is the entry of the extent added below the external span. |
parameters: |
|
history: |
Introduced in version 1.4.0 |
example: |
|
notes: |
|
related: |
Trace.EndTrace()
method: |
Trace.EndTrace() |
---|---|
description: |
End a trace. |
returns: |
An event that can be populated with key/value pairs and reported. Similar to Trace.StartTrace(), most of the time requests will egress your application stack at your webserver and so our instrumentation will take care of ending traces. But you will need this method for tracing across external spans. |
parameters: |
|
history: |
Introduced in version 1.4.0 |
example: |
|
notes: |
|
related: |
Trace.EndTraceHttpService()
method: | Trace.EndTraceHttpService() |
---|---|
description: | End a trace. |
returns: | An event that can be populated with key/value pairs and reported. Similar to Trace.StartTrace(), most of the time requests will egress your application stack at your webserver and so our instrumentation will take care of ending traces. But you will need this method for tracing across external spans. |
parameters: |
|
history: | Introduced in version 4.3.1 |
example: |
|
notes: |
|
related: |
Trace.CreateEntryEvent()
method: |
Trace.CreateEntryEvent() |
---|---|
description: |
Start a new span. It's up to you, the application developer, to decide how to segment your application modules and sub-systems into spans. |
returns: |
An event that can be populated with key/value pairs and reported. |
parameters: |
|
history: |
Introduced in version 1.4.0 |
example: |
|
notes: |
|
related: |
Trace.CreateExitEvent()
method: |
Trace.CreateExitEvent() |
---|---|
description: |
Complete the specified span. |
returns: |
An event that can be populated with key/value pairs and reported. |
parameters: |
|
history: |
Introduced in version 1.4.0 |
example: |
|
notes: |
|
related: |
Trace.CreateInfoEvent()
method: |
Trace.CreateInfoEvent() |
---|---|
description: |
Info events enable you to report information in between the entry and exit events of a span, particularly for special interpretation. |
returns: |
An event that can be populated with key/value pairs and reported. |
parameters: |
|
history: |
Introduced in version 1.4.0 |
example: |
|
notes: |
|
related: |
Trace.ReportException()
method: |
Trace.ReportException() |
---|---|
description: |
Report an exception, including a backtrace. |
history: |
Introduced in version 1.4.0 |
example: |
See report errors. |
notes: |
|
Trace.SetTransactionName()
method: |
Trace.SetTransactionName() |
---|---|
description: |
The .NET agent out-of-the-box instrumentation assigns a transaction name based on URL and Controller/Action values detected. However, you may want to override the transaction name to better describe your instrumented operation. If multiple transaction names are set on the same trace, then the last one would be used. |
returns: |
A |
parameters: |
|
history: |
Introduced in version 3.3.0 for .NET Framework SDK. Introduced in version 3.3.2 for .NET Core SDK. |
example: |
|
notes: |
|
Trace.IsAgentAvailable()
method: | Trace.IsAgentAvailable() |
---|---|
description: | Method can be used to determine if the .NET agent is available. |
returns: | A bool indicating if .NET agent is available. |
history: | Introduced in version 3.4.0 for .NET Framework SDK. |
Trace.WaitUntilAgentReady(timeout)
method: |
Trace.WaitUntilAgentReady() |
---|---|
description: |
Blocks until agent is ready (established connection with data collector) or timeout expired. |
parameters: |
|
returns: |
A |
history: |
Introduced in version 3.4.0 for .NET Framework SDK. |
Trace.WaitUntilAgentReady(timeout, statusCode)
method: |
Trace.WaitUntilAgentReady() |
---|---|
description: |
Blocks until the agent is ready (established connection with data collector) or timeout expired. |
parameters: |
|
returns: |
A |
history: |
Introduced in version 3.4.0 for .NET Framework SDK. |
EntryTraceEvent.SetAsync()
method: |
EntryTraceEvent.SetAsync() |
---|---|
description: |
Declare the span initiated by the entry event as asynchronous. See tracing multi-threaded applications for usage information. |
history: |
Introduced in version 1.4.0 |
example: |
|
ExitTraceEvent.AddEdge()
method: |
ExitTraceEvent.AddEdge() |
---|---|
description: |
Associate an external span with the current trace. See tracing across external spans for usage information. |
parameters: |
|
history: |
Introduced in version 1.4.0 |
example: |
|
.AddInfo()
method: | .AddInfo() |
---|---|
description: | Attach information to events in the form of key/value pairs particularly for special interpretation, but also to attach any other desired information. |
history: | Introduced in version 1.4.0 |
example: | See add info events. |
notes: | Duplicate keys are ignored, and keys may not be null. |
.AddBackTrace()
method: |
.AddBackTrace() |
---|---|
description: |
To report the back traces of the certain time frame of the application. This adds the back trace of the current thread to the event. |
history: |
Introduced in version 1.4.0 |
example: |
|
.Report()
method: |
.Report() |
---|---|
description: |
Use this method to report events, as they are not automatically reported upon creation. |
history: |
Introduced in version 1.4.0 |
example: |
|
.ReportAndReturnTraceId()
method: |
EndTraceEvent.ReportAndReturnTraceId() |
---|---|
description: |
Report the end trace event and return the trace id. See tracing across external spans for usage information. |
history: |
Introduced in version 1.4.0 |
example: |
|
related: |
.SummaryMetric()
method: |
.SummaryMetric() |
---|---|
description: |
Use this method to report a metric value. The metric values reported are aggregated and flushed every 60 seconds. |
parameters: |
|
history: |
Introduced in version 3.0.0 |
example: |
|
.IncrementMetric()
method: |
.IncrementMetric() |
---|---|
description: |
Use this method to report the number of times an action occurs. The metric counts reported are summed and flushed every 60 seconds. |
parameters: |
|
history: |
Introduced in version 3.0.0 |
example: |
|
Trace.GetCurrentTraceId()
method: | Trace.GetCurrentTraceId() |
---|---|
description: | Get the current trace id, which can be passed to an external process. This method should not be used for any purpose other than for tracing across external spans. |
returns: | A string representing the current trace or an empty string if no trace is active. |
history: | Introduced in version 1.4.0 |
Trace.GetCurrentLogTraceId()
method: |
Trace.GetCurrentLogTraceId() |
---|---|
description: |
Get a compact form of the current trace id suitable for logging purpose. This method should not be used for any purpose other than for logging. |
parameters: |
|
returns: |
A string representing a compact form of the current trace id or an empty string if no trace is active. |
history: |
Introduced in version 3.8.0 |
TraceContext.GetCurrent()
method: | TraceContext.GetCurrent() |
---|---|
description: | Get the current trace id, which can be passed to a background/child thread. See tracing multi-threaded applications for usage information. |
returns: | A string representing the current trace. A trace ID should always be returned even if the request isn't being sampled. |
history: | Introduced in version 1.4.0 |
TraceContext.ClearCurrentContext()
method: | TraceContext.ClearCurrentContext() |
---|---|
description: | Unset the trace id before returning a thread back to the threadpool. See tracing multi-threaded applications for usage information. |
history: | Introduced in version 1.4.0 |
TraceContext.SetAsCurrent()
method: | TraceContext.SetAsCurrent() |
---|---|
description: | Set the trace id for a background/child thread. See tracing multi-threaded applications for usage information. |
history: | Introduced in version 1.4.0 |
Troubleshooting
See the troubleshooting page for more information.
The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.