SecureFlow Manager Filter Specification

Unblu and some implementation partners provide filters for many integration scenarios (see this page for further information). Sometimes, however, an adapted integration may be required. This custom filter specification describes how a filter can be implemented.

Depending on the integration method, the filter can be implemented thus:

This document will not explain all integrations and implementation variations in detail. Instead it will focus on providing all necessary information to allow developers to implement a fitting solution on their own.

With the following sections, requirements are split into the responsibilities of the filter:

If the filter is integrated into or implemented as a proxy or if the filter is integrated into the customer’s web or application server, it must

If the filter is running in the end user’s browser, it must

The filter must have access to all HTTP(S) requests and responses that are exchanged between the browser and the backend server. It can be implemented as a reverse proxy or as a filter directly running on the customer’s web or application server or the end user’s browser. It must be able to intercept requests, forward response bodies to Unblu and/or modify responses (headers and content). In addition it must be able to redirect certain requests (messages from the browser) to the collaboration server instead of letting the backend process them.

When implemented as a proxy, the filter must be able to

In order to keep the configuration in one place, the filter needs to be configured dynamically from the collaboration server. Local configuration should be limited to things that cannot be provided by the server (such as locating the collaboration server) and to purely implementation-specific, technical configuration. When the filter starts up, it must read its configuration from the server. The configuration contains all information the filter needs in order to do its work. Because the collaboration server might be reconfigured without the filter being restarted, a special header containing a configuration version identifier is included in every response Unblu sends to the filter. As soon as the version identifier changes, the filter must reload the configuration from the server.

The configuration defines a list of path prefixes (i.e. /unblu/) that identifies requests that need to be forwarded to the collaboration server. All requests from the customers browser starting with this prefix must be forwarded to the server instead of processing them as normal requests to the backend web or application server.

The filter must send the content of the response bodies (enriched by additional information) to the collaboration server so that it can be stored for (later) playback in the Unblu player. The filter has to decide whether a response needs to be sent to the collaboration server according to rules defined in the configuration.

The filter must be able to inject JavaScript and CSS code into the body of the HTML response. The content that needs to be injected and the rules defining whether a response needs injection or not are defined in the configuration.

The filter must be able to perform regular expression replacements in textual HTTP responses (HTML, CSS, JavaScript…​). A list of replacements is included in the filter configuration.

Parts of the injected code need to be dynamic (for instance the id of the current cache content). Therefore, the filter needs to perform simple string replacements in the JavaScript content before injecting into the HTML body of the response. These string replacements are based on the #Environment Variables that the filter provides.

The decision whether or not a page needs to be sent to the collaboration server or whether it needs code injection must be taken on the filter. A round-trip to the collaboration server is not acceptable. Therefore, the filter needs to evaluate a set of rules (provided in the configuration) in order to decide. These rules need to be able to take into account the #Environment Variables.

The filter needs to provide a predefined set of environment variables to be used in #String replacement and #Rule evaluation. Some of these environment variables are static (such as unbluPublicPath, the path to the collaboration server), others are dynamic and need to be provided in a request scope (requestUri, cookie values, contentType…​)

The filter must smoothly continue to deliver backend requests even if the collaboration server is (temporarily) unavailable.

UML overview of the Java implementation

Filter implementations must provide environment variables and string substitution functionality. This functionality is used by the rule evaluation system and by the code injection.

Variable names must be case insensitive.

Environment variables must be provided on three different scopes: filter, configuration and request. In the filter scope, globally valid variables such as the URL of the collaboration server are stored. The request scope holds variables that are only valid for a single request (cookie values, response content type…​). The request scope environment must fall back on the configuration scope environment if a variable key is not found within the request scope, configuration scope must fall back to filter scope.

Filter implementations must be able to search for placeholders in strings and replace them with variable values from the environment. The expansion of placeholders with their value must be performed recursively as variable values can also hold placeholders.

Placeholders have the following format: ${PLACEHOLDER_NAME}. Only 'a' to 'z' characters, - and _ are allowed in variable keys. The following regular expression can be used to find placeholders: \${a-zA-Z_-+}.

Lookup of placeholders is case insensitive.

Environment variables are defined from several sources:

The following table lists all environment variables that filter implementations must support:

Filter implementations must provide rule evaluation support. Rules are boolean expressions that consist of various comparisons (equal, greater than, startsWith…​), conjunctive ("and") and disjunctive ("or") combinations and negations ("not"). Rules evaluations are used where the filter needs to decide whether a response needs to be sent to the collaboration server for caching and to decide if a code injection needs to be performed or not.

Rule evaluation always takes place within an environment scope (filter, configuration or request). The evaluation of every rule within an environment results in a boolean value.

Comparison

type-property: comparison

Comparison rules consist of the following parts:

The operator property can have one of the following values:

| operator | true if | | :-- | :-- | | equals | left and right side are equal (string comparison) | | startsWith | left side string value starts with right side string value | | endsWith | left side string value ends with right side string value | | contains | left side string value contains right side string value | | = | left side is equal to right side (numeric comparison) | | > | left side is bigger than right side | | < | left side is smaller than right side | | >= | left side is bigger or equal to right side | | <= | left side is smaller or equal to right side | For example, a comparison rule could compare the response content type of a request against a string literal:

type-property: and

And rules consist of a list of compound rules ("rules" property).

And rules evaluate to true if all components evaluate to true.

type-property: or

Or rules consist of a list of compound rules ("rules" property).

Or rules evaluate to true if at least one of the components evaluates to true.

type-property: not

Not rules consist of a single rule ("rule" property).

Not rules evaluate to true if the contained rule evaluates to false.

UML of the Java implementation

The filter configuration holds a list of ConditionalCodeInjections. A conditional code injection consists of a Rule (condition) and a list of CodeInjections that must be performed if the condition evaluates to true (in the request scope environment).

Every CodeInjection consists of a reference (place in the HTML code where the injection must be performed), a type (internal or external JavaScript, internal or external style sheet) and a value holding the string value of the code injection. The value can hold placeholders so it must be expanded in the request environment prior to injection.

The code injection references represents the spot in the HTML code where the injection string must be placed.

The code injection type defines how the injection string must be modified before it is inserted in the HTML.

The filter configuration holds a list of ConditionalRegexReplacement instances. Every ConditionalRegexReplacement holds a rule (condition property). If this condition evaluates to true in the request scope environment the regular expression-based string replacement must be performed on the response body. All matches of the given pattern must be replaced with the given replacement string. The replacement must be expanded in the request scope environment as it can contain placeholders.

Filter implementations need to communicate with the collaboration server in the following situations:

In the first case the filter simply proxies incoming requests to the server by translating the request path and forwarding query string, body and parts of the headers.

For the other cases (read configuration, cache contents), the server provides a simple HTTP interface.

The filter has to forward all requests that have a path that starts with ${UNBLU_PUBLIC_PATH} to the collaboration server.

For instance, if the filter is reachable at http://foo.com/ and the Unblu server URL is http://localhost:8080/unblu/ and ${UNBLU_PUBLIC_PATH} is /unblu then a request to http://foo.com/unblu/js/bar.jshas to be forwarded to http://localhost:8080/unblu/js/bar.js. The request to the collaboration server has to include the original query string and the body of the request (if it is a POST request). When forwarding a request to the server generally all request headers (as sent from the browser) must also be set in the request to the server and all response headers (as sent from the server) must also be set in the response to the browser.

In most cases a forwarding request to Unblu is realized using an already in place reverse proxy (i.e., Apache mod_proxy or similar). The proxy must at least implement the "GET" and "POST" HTTP methods.

When the filter reads configuration from the collaboration server or sends a response to the server for caching, it uses a simple HTTP interface.

The HTTP interface is reachable through the URL ${UNBLU_SYSTEM_URL}/filterBackend.

Requests to this interface consist of a set of key/value pairs. These pairs can be sent in the form of proprietary headers (x-unblu-<name>:<value>) or as request parameters in the query string (i.e., http://localhost:8080/unblu/rest/filterBackend?x-unblu-action=add-to-cache…​). In addition to proprietary headers or query string parameters, binary data (body of responses to be cached) is transmitted in the body of a post request. If data is transmitted content-type has to be set to application/octet-stream and the content-length header must be set.

Responses from the HTTP interface always consist of a JSON string holding either a response (depending on the action) or an error message.

In addition to the response body, the HTTP interface responses include the following proprietary headers:

The following parameters must be set in every request to the HTTP interface:

Depending on the "action" parameter additional parameters are required:

This section describes the actual work a filter has to do. A filter implementation must intercept all requests (whether as a module in the HTTP server or as a proxy). For every request the following steps need to be performed (pseudo code).

If the collaboration server is (temporarily) unavailable, the filter must not disturb the normal operation of the backend HTTP server. In order to achieve this it might make sense to remember the state of the server within the filter. Doing this you can avoid running into timeouts at every request.

This could be done similar to:

As soon as the filter fails to call the collaboration server it starts a background thread that waits for the server to come up again. Until the server is up again the filter does not attempt to send requests to the server anymore. As soon as Unblu is up again the filter starts sending request to it again. In order to check if the server is up the "ping" action of the HTTP interface can be used.

This section describes the life cycle of a filter.