Collection Filtering

Apiture APIs collections represent possibly large sets of resources. A user may have thousands of transactions. Filtering allows the client to request a subset of items from from the collection that satisfy specific Boolean expressions called filters. The filter expression is applied in the context of the individual resources in the collection and can compare properties of that resource to other properties or to constant values or or other complex expressions.

For example, a transaction contains several filterable properties:

  • the date the transaction was processed
  • the type and subtype of the transaction
  • an amount object, with value and the currency for that transfer amount

Simple filtering

The simplest form of filtering is exact comparisons. These can be done with simple query parameters on the GET operation to the collection, such as:

  GET /transactions/pastTransactions?amount.value=210.50
  GET /transactions/pastTransactions?date=2017-10-02&type=debit

As a convenience, the vertical bar character | may be used to separate choices:

  GET /transactions/scheduledTransactions?state=inactive|pending

This means select items where state is either inactive or pending.

Complex filtering

Simple filtering can only do exact equality matching. If a client wants to perform other comparisons such as less than, greater than, range checking, or inequality, use filter criteria with the ?filter=filter-criteria query parameter.

Using ?filter=filter-criteria, one may perform more complex filtering, not just exact matches. This can involve ranges (for numbers or dates), relational operators such as less than, greater that or equal to, set inclusion, starts with a substring, ends with a substring, contains a substring, etc.

There are many possible filter query expression syntaxes. The Apiture API filter syntax is not be tied to a specific implementation technology (such as SQL WHERE clauses or ODATA queries). It is unambiguous, expressive, and extensible. Infix notation such as a < b requires defining precedence rules which is often complicated, verbose, and varies from language to language. Clients, especially expression generators/builders, tend to put parentheses around most subexpressions anyway. For example, precedence rules dictate that a + b * -c is evaluated as (a + (b * (-c))).

Most infix operators do not fit in URLs, as they need to be URL encoded: Special characters like spaces or the relational operators < and > must be encoded as %20, &lt; and &gt; in HTML. The infix expression a<b and b>c must be URL encoded as a%3Cb%20and%20b%3Ec which is not very readable in URIs. The & character cannot be used for and because it is a URL special character used to separate query parameters.

Prefix function notation, such as f(arg0, arg1, ....), avoids these problems by using alphanumeric identifiers for all operators. a == b && b <= c is expressed as and(eq(a,b),le(b,c)). In query parameters, the (, ,, and ) characters do not have to be URL encoded as per RFC 3986. This also parses directly into an abstract syntax tree and is thus easy to generate from clients without injecting explicit paretheses to control expression priority. In addition, this notation also allows defining date, time, and date-time literals. For example, the date literal 2017-01-10 is not ambiguous: it is not 2017 minus 1 minus 10, because there is no infix - operator. The function notation also allow concise varargs operations: eq(a,b,c,100) for a == b && b == c && c== 100 or le(100,b,200) for range checks such as 100 <= b and b <= 200.

Most functions used in filtering are predicates: functions which return Boolean results.

In the filter functions below, ... is used to denote varargs and [,arg] denotes that an argument is optional.

Filter functions

Function Description  
eq(a,b,...) true iff all values are the same type and equal each other
Examples:
eq(request,"deposit")
eq(balance, 0)
 
ne(a,b) true iff a is not equal to b. varargs not supported because of the combinator blowup or ambiguity
Examples:
ne(0,balance)
 
lt(a,b,...) true iff a strictly less than b; all values must have the same type (either string, number, date, time, or date-time)
Examples:
lt(debit,10000)
lt(birthDate,2000-01-01)
 
le(a,b,...) true iff a less than or equal to b ; all values must have the same type (either string, number, date, time, or date-time)
Examples:
le(debit,10000)
le(10000,debit,20000)
 
gt(a,b,...) true iff a strictly greater than b ; all values must have the same type (either string, number, date, time, or date-time)
Examples:
gt(rate,0.5)
 
ge(a,b,...) true iff a greater than or equal to b ; all values must have the same type (either string, number, date, time, or date-time)
Examples:
ge(debit,25000)
ge(birthDate,1996-01-01)
 
in(a,v0,v1,...) true iff a is in the set of values v0,v1,…; works for string/strings or number/numbers.
Examples:
in(option, "M", "F")
Note that the form in("key",a,b,c) is a concise way of expressing the expression
or(eq(a,"key"),eq(b,"key"),eq(c,"key")) and the form in(state,'active','inactive','pending') is equivalent to or(eq(state,"active"), eq(state,"inactive"), eq(state,"pending"))
 
matches(s,regex[,flags]) true iff a matches the regular expression regex. flags are regular expression flags; "i" means ignore case
Examples:
matches(ph, "^\d{3}-\d{3}-\d{4}$")
 
startsWith(a,prefix[,flags]) true iff a starts with the prefix string prefix regular expression. flags are regular expression flags; "i" means ignore case
Examples:
startsWith(s, "<script")
 
endsWith(a,suffix[,flags]) true iff a ends with the suffix string suffix regular expression. flags are regular expression flags; "i" means ignore case
Examples:
endsWith(s, ".png")
 
contains(s,substring) true iff a (a string value) contains an exact substring
Examples:
contains(s, "Oak")
 
search(substring) true iff the resource contains a substring within the text properties of the resource’s representation
Examples:
search("oak")
 
now() The current date-time  
today() The current date  
time() The current time of day  
time(date-time-value) Returns the time of day portion of a date-time value
Examples:
time(createdAt)
time(2018-01-10T05:40:07.375Z) returns the time value, 05:40:07.375
 
date(date-time-value) Returns the date portion of day of a date-time value
Examples:
date(createdAt)
date(2018-01-10T05:40:07.375Z) returns the date value 2018-01-10
 

Literals

Literals (constants) may be used in filter expressions as arguments to predicate functions:

Type Description
numbers integer and floating point literals
Examples:
0
100.0
-25000
strings Sequences of Unicode characters, enclosed in either double quotes or single quotes may be used.
To include the quoting character, repeat it twice (the last two are equivalent).
Examples:
"Oak"
'A string with a "nested string" in it'
'It''s a trap!'
"It's a trap!"
time An RFC 3339 time literal value consisting of hh:mm or hh:mm:ss
Examples:
15:00
00:00:00
date An RFC 3339 calendar date value consisting of YYYY-MM-DD
Examples:
1998-01-01
date-time An RFC 3339 time value consisting of YYYY-MM-DDThh:mm:ssZ where Z is either the code Z for GMT (UTC) or a +hh:mm time zone offset
Examples:
2018-01-12T06:59:17.375Z
2018-01-12T06:59:00+05:00

Identifiers

When filtering a collection, an identifier in a filter expression is the name of a property in the resource in that collection, such as (for the transfer resource): date, amount, and type.

If a resource has an nested object, a compound identifier uses name the containing object, a separating period (.), and the nested property name. For example, for a product type query may reference an embedded amount.value.

In Apiture APIs, all property names use only ASCII alphanumeric characters.

Objects inside the HAL _embedded structure are not queriable, as these are not actually part of the resource.

Text Searching

Collections also support free text searching as a means to filter the collection to resources that contain a seach string.

Combining filters

When two or more of these forms are combined they are joined with an implicit and operator.

For example, the query

   GET /products/productTypes?state=active&subtypeCount=0&q=demand&filter=ge(createdAt,datetime)

has the net effect of combining the query elements, as if the request was:

   GET /products/productTypes?filter=and(eq(state,'active'),eq(subtypeCount,0),ge(createdAt,datetime),search('demand'))

[ Note: We may make &filter= and &q= mutually exclusive, so the above might not be allowed. TBD. ]