Filtering from a subset of predicates

To search for jobs after finishing university I build a job board scraper. The backend (F#) of this site scraped the individual job listings, and a frontend (Vue.js) showed the jobs and allowed filtering by the attributes scraped by the backend.

One of my favourite parts of the system is implementation of the filtering mechanism.

Naive implementation

The first two filters were binary, like a checkbox they were either on or off. This meant the “state space” of enabled filters was small, 2^2 = 4. This corresponds to either applying no filter, one filter, or both. This was easy enough to cover with four if/else-if/else blocks, where each block would apply the correct filtering and return.

Then I added a more complex filter, a string search over the location field. To make it a bit more usable, I also added a checkbox allowing the location search query to be treated as a regular expression. This expanded the “state space” of the filters to be 2^3 = 8, which was too large to use branching to apply the correct combination of filters, especially if I wanted to add more later.

At this point, the exponential growth in the filters was apparent. I wanted the filtering code to grow linearly in the number of filters, doubling the lines of filtering code each time I wanted to add a new filter was clearly not going to achieve this.

A better implementation

To solve this I changed the way the active filters were calculated and applied.

All possible filters are represented as a list of objects of type: { condition: boolean; predicate: Job => boolean }. The condition property is a boolean value describing whether or not to apply the filter considering the current page state (which is stored as a global variable). The predicate property stores a function which when given a job, returns whether or not it should be included in the filtered results.

Note that this one of my side-projects, so the Vue code might be a bit scrappy.

The allFilters list is computed to allow the condition properties to be automatically updated as the page state changes.

const allFilters = computed(() => [
  {
    condition: selectedCompanies.value.length > 0,
    predicate: j => selectedCompanies.value.includes(j.company)
  },
  {
    condition: showOnlyUnseen.value,
    predicate: j => !j.seen
  },
  {
    condition: locationFilterSettings.value.filter != "" && locationFilterSettings.value.regex,
    predicate:  j => new RegExp(locationFilterSettings.value.filter).test(j.location)
  },
  {
    condition: locationFilterSettings.value.filter != "" && !locationFilterSettings.value.regex
    predicate: j => j.location.includes(locationFilterSettings.value.filter)
  }
  {
    condition: dateFilter.value != null,
    predicate: j => (new Date(j.published_date)) >= dateFilter.value
  }
]);

function calculateDisplayedJobs() {
  let predicates = allFilters.value.filter(c => c.condition).map(c => c.predicate);
  jobsToDisplay.value = jobs.filter(job => predicates.every(pred => pred(job)))
}

The calculateDisplayedJobs function updates the value of the global variable jobsToDisplay based on the selected filters.

First, it filters the list of filters to obtain any which have their condition set to true, this gives a list of filters which should be applied given the current page state. Then, the predicate property is extracted from each active filter. Then, the jobs list is filtered to contain only jobs which satisfy all of the predicates. This list will contain all jobs which satisfy the user’s selected filters, completing the filtering process.

To add a new filter you only need to add an entry in the allFilters list, giving the condition of the page state for the filter to be active, and providing a predicate which applies the filter. The calculateDisplayedJobs function does not need to be updated. This achieves the goal of linearly scaling the size of the codebase by the total number of filters.

To make it a bit more elegant, the allFilters list could be made not-computed, and condition property could be modified to have the type () => boolean. Then the calculateDisplayedJobs function could invoke the condition function to determine which filters should be active, rather than the relying on the computed property.

function calculateDisplayedJobs() {
  let predicates = allFilters.value.filter(c => c.condition()).map(c => c.predicate);
  jobsToDisplay.value = jobs.filter(job => predicates.every(pred => pred(job)))
}

Overall, I think this filtering system is good and it provides a low-maintenance way of applying a combination of filters dependent on page state.