QueryFilter: A Model Filtering Concept

Having a clean code with single responsibility is important, and doing that for model filtering can be easy and very powerful. Believe me.

This blog post is related to a talk I gave on April, 2018 on Darkmira Tour PHP 2018, in Brasília/DF, Brazil, with the same title. The slides are on SpeakerDeck.

Filtering models was, for a very long time, a hard task for me. I admit that I could not think in some easy way to do that. I tried, refactored some code, created custom classes for that, but I never thought how this could be easily implemented.

Watching a Laracast’s video from 2016 about the Laravel’s Eloquent ORM I faced of with a bunch of classes and a trait that removed a lot of trash from my controller actions. That was called by Jeffrey Way the QueryFilter.

QueryFilter is a very simple refactor, but it can bring a lot of flexibility to your code, having separate logics responsibilities and the opportunity to focus on custom filter behaviors. I call it a “concept” and because of that it can be applied to any programming language or any framework. Here I’m gonna show you the concept, using the PHP Laravel framework as example.

The concept

The QueryFilter concept, like the name says, is responsible for filtering based on the URL query. We’re taking about filtering models according to the query string you have.

GET /posts?title=source&status=publish

Using this example, we’d like to filter posts (Post model):

• Having the word “source” in its title column;
• With “publish” status column.

Example app

Let’s create very simple Laravel app to illustrate how it works. This app is only responsible for showing posts in the GET /posts URL, returning a post collection in JSON format. I’m gonna use my own blog as data source, with a WordPress database, and my open-source project Corcel, that brings the ability to map WordPress database tables to Laravel models, in a very easy way.

In this app we’re using Corcel only for retrieving posts from the database, playing with filters using the QueryFilter concept. For more information about Corcel please take a look on the documentation page on GitHub.

After installing Corcel – composer require jgrossi/corcel -, let’s create a custom App\Post class extending Corcel’s one:

namespace App;

use Corcel\Model\Post as CorcelPost;

class Post extends CorcelPost
{
    /**
     * @var string
     */
    protected $postType = 'post';
}

Here I’m saying that the App\Post class is gonna return only post post type, not page or anything else.

The GET route:

Route::get('/posts', 'PostController@index');

And the Controller class with a index() action that will be responsible for returning posts in JSON format.

namespace App\Http\Controllers;

use App\Http\Resources\Post as PostResource;
use App\Post;

class PostController extends Controller
{
    /**
     * @return \Illuminate\Http\Resources\Json\AnonymousResourceCollection
     */
    public function index()
    {
        $posts = Post::limit(10)->get();

        return PostResource::collection($posts);
    }
}

The App\Http\Resources\Post class, retuning only the JSON
fields I want to:

namespace App\Http\Resources;

use Illuminate\Http\Resources\Json\JsonResource;

class Post extends JsonResource
{
    /**
     * Transform the resource into an array.
     *
     * @param  \Illuminate\Http\Request  $request
     * @return array
     */
    public function toArray($request)
    {
        return [
            'id' => $this->ID,
            'title' => $this->post_title,
            'slug' => $this->post_name,
            'status' => $this->post_status,
            'type' => $this->post_type,
            'published_at' => $this->post_date,
            'updated_at' => $this->post_modified,
        ];
    }
}

WordPress wp_posts‘s table has much more columns, but for an easier understanding about what’s going on I’m returning just a few fields, excluding for example, the most important one, the post_content.

This app’s domain is query-filter.test, so when visiting http://query-filter.test/posts` you should see a JSON response with some posts. Remembering in this case I’m using my own Blog’s database table, but this should work with any blog database in WordPress.

In this first call we have posts with many different status, like publish, auto-draft, draft, etc. Now it’s time to start the filtering process, let’s filter first by the status field.

The basic filtering

The first thing that comes in mind is to inject a Request instance in the controller action and filter the post status by what we have in the query string (URL). Usually you can do something like this:

/**
 * @param Request $request
 * @return \Illuminate\Http\Resources\Json\AnonymousResourceCollection
 */
public function index(Request $request)
{
    $query = Post::limit(10);

    if ($request->filled('status')) {
        $query->where('post_status', $request->get('status'));
    }

    $posts = $query->get();

    return PostResource::collection($posts);
}

Then if you go to the following url http://query-filter.test/posts?status=publish you should see only published posts, with publish status. You can do the same for the post title, for example, using a bit different logic:

if ($request->filled('status')) {
    $query->where('post_status', $request->get('status'));
}

if ($request->filled('title')) {
    $title = $request->get('title');
    $query->where('post_title', 'like', "%$title%");
}

The problem here is that your controller action is getting even bigger as the many fields you need to filter by. So its time to use the QueryFilter concept and remove the responsibility from the Controller class.

Implementing QueryFilter

The easy way to understand this filtering concept is that you will have a filter class for each resource you have, matching the methods with each field you would like to filter by. Remember the URL we have:

GET /posts?title=source&status=publish

Once we’re talking about posts, we will have then a PostFilter class with title() and status() methods, just to start. This PostFilter class should extend a abstract QueryFilter class, that’s responsible for the methods calling process.

The QueryFilter class will receive a Request instance – to get the fields and values from – and an Eloquent Builder class, the same we use in the controller with where() clauses. This Builder class will be used inside any “field method” to filter field by field using a specific logic.

Just to make things easier, we’re now working on the App\Http\Filters namespace, so the QueryFilter class is stored on App\Http\Filters\QueryFilter.php.

The apply() method

The QueryFilter class should have a method to be responsible for calling all fields’ filters we want, in the child class (PostFilter). The apply() method receives a Builder instance, gets all fields from the Request, checks if there’s a method called $field(), and if true, calls that method sending the field value as parameter.

For example, for the title() method, the apply() method is gonna call it ->title($title), in our example case ->title('source'):

namespace App\Http\Filters;

use Illuminate\Database\Eloquent\Builder;
use Illuminate\Http\Request;

abstract class QueryFilter
{
    /**
     * @var Request
     */
    protected $request;

    /**
     * @var Builder
     */
    protected $builder;

    /**
     * @param Request $request
     */
    public function __construct(Request $request)
    {
        $this->request = $request;
    }

    /**
     * @param Builder $builder
     */
    public function apply(Builder $builder)
    {
        $this->builder = $builder;

        foreach ($this->fields() as $field => $value) {
            $method = camel_case($field);
            if (method_exists($this, $method)) {
                call_user_func_array([$this, $method], (array)$value);
            }
        }
    }

    /**
     * @return array
     */
    protected function fields(): array
    {
        return array_filter(
            array_map('trim', $this->request->all())
        );
    }
}

The main point here is, for each field we have in the Request instance we’ll have a specific method in the filter class (child one), allowing you to customize each field filter logic.

The PostFilter class

Now that we have the abstract QueryFilter class created we must implement a new class to extend that parent one. So, once we are talking about posts resources let’s call this class PostFilter: App\Http\Filters\PostFilter.php.

Like we said before, this class must have a method for each field we would like to filter by. So for the given example: GET /posts?title=source&status=publish we must have two method, called ->title($value) and ->status($value), respectively. Each method will receive the current value from the query string, in this case the “source” string and the “publish” string, respectively.

namespace App\Http\Filters;

use Illuminate\Database\Eloquent\Builder;

class PostFilter extends QueryFilter
{
    /**
     * @param string $status
     */
    public function status(string $status)
    {
        $this->builder->where('post_status', strtolower($status));
    }

    /**
     * @param string $title
     */
    public function title(string $title)
    {
        $words = array_filter(explode(' ', $title));

        $this->builder->where(function (Builder $query) use ($words) {
            foreach ($words as $word) {
                $query->where('post_title', 'like', "%$word%");
            }
        });
    }
}

In this case, the status() method is looking for exactly the same string in the $status variable, and the method title() is exploding the $title variable by spaces, to be able to filter by each word we find in the query string.

The important point here is not the logic inside each method, but once you have a specific method for each field you can implement any logic, complex or simples, using third-party tools or not. The point is that you have now a specific class for filtering, so its responsibility is just filtering, then play with that.

The filter() scope

All filtering process is related to a model builder in this case. So we have to find a way to call the filter’s ->apply($builder) method from, sending to it the model builder class you want to filter.

Laravel allows us to create specific scopes for any model we have. Using the scopeMethodName() convention you’ll be able to have a ->methodName() scope, receiving by default the current model Builder instance.

Using the concept we want, let’s create a filter() scope inside the Post model we have:

/**
 * @param Builder $builder
 * @param QueryFilter $filter
 */
public function scopeFilter(Builder $builder, QueryFilter $filter)
{
    $filter->apply($builder);
}

In this case, we just created the Post::filter() scope, that should receive a QueryFilter instance, in our case the PostFilter one:

$filteredPosts = Post::filter($postFilter)->get();

So all filtering logic is being handled by the filter($postFilter) call, avoiding giving the controller the extra work of filtering models, what’s not its responsibility.

For better code reuse, let’s move the scopeFilter() method to a Trait (App\Http\Filters\Filterable.php), allowing us to reuse that code by just using it for each model we would like to be able to filter:

namespace App\Http\Filters;

use Illuminate\Database\Eloquent\Builder;

trait Filterable
{
    /**
     * @param Builder $builder
     * @param QueryFilter $filter
     */
    public function scopeFilter(Builder $builder, QueryFilter $filter)
    {
        $filter->apply($builder);
    }
}

So now our Post model is just:

namespace App;

use App\Http\Filters\Filterable;
use Corcel\Model\Post as CorcelPost;

class Post extends CorcelPost
{
    use Filterable;

    /**
     * @var string
     */
    protected $postType = 'post';
}

Method-injecting the PostFilter class

Now that we have all the QueryFilter logic implemented the only thing that we have to do is to method-inject a PostFilter instance in our PostController::index() action and call the filter() scope we created for the Post model.

class PostController extends Controller
{
    /**
     * @param PostFilter $filter
     * @return \Illuminate\Http\Resources\Json\AnonymousResourceCollection
     */
    public function index(PostFilter $filter)
    {
        $posts = Post::filter($filter)->limit(10)->get();

        return PostResource::collection($posts);
    }
}

That’s all. Now we move all the filtering logic to a proper class, respecting the Single Responsibility Principle, the S from the SOLID principle.

Sorting improvement

Once we are playing with the query string, we can extends the filter logic to something like a “sort” action. So we can have a bit different query string like this one, wanting to filter and sort models:

GET /posts?title=source&status=publish&sort=-post_title, post_content

In this case we would like to sort posts first by post_title column descending and then by post_content ascending. Is it clear? How can we implement that? Let’s add a sort() method to the QueryFilter class, so if there’s a “sort=foo” in the query string the sort() method is gonna be called and the builder change to order models, for example.

/**
 * Sort the collection by the sort field
 * Examples: sort= title,-status || sort=-title || sort=status
 *
 * @param string $value
 */
protected function sort(string $value)
{
    collect(explode(',', $value))->mapWithKeys(function (string $field) {
        switch (substr($field, 0, 1)) {
            case '-':
                return [substr($field, 1) => 'desc'];
            case ' ':
                return [substr($field, 1) => 'asc'];
            default:
                return [$field => 'asc'];
        }
    })->each(function (string $order, string $field) {
        $this->builder->orderBy($field, $order);
    });
}

We are telling the builder that if the first char is it should order by ASC and if - by DESC. The collection usage is necessary only for sorting by multiple fields, separate by a , char. Play with that!

Conclusion

The QueryFilter concept is very handful for me, allowing me to extract extra responsibility from the controller classes. This even makes easy to write tests, because you can have simple unit tests that test just the Builder content according to the query string.

I can’t imagine myself working with filters based on the query string without thinking in the QueryFilter concept. The best part is that you can play and extend its usage with anything, even ElasticSearch, or Solr, for example.

Think that filtering is not your controller’s responsibility anymore, but the XFilter classes you need to create. It’s easier to think this way, and I’m sure your controller code is gonna be much more cleaner with this approach.

Here in this post we have implemented all the necessary code to make everything working, because like I said before, it’s a concept. In my opinion is important to reimplement this by hand to understand how it works properly, but of course you can use some composer packages that already do something similar.

I usually use this composer package: https://packagist.org/packages/kblais/query-filter from Killian Blais, which I inclusive has contributed a bit to, introducing some specific features and fixing some bugs. You can also try another one: https://packagist.org/packages/cerbero/query-filters.

The point here is that the QueryFilter is a concept, so you can use it like you want to, even changing its behavior to something different.

This post worked with PHP and Laravel but you’re welcome to work with any language or framework, creating useful and interesting packages. Enjoy!