Omega Search / Progressive Decoupling

This content was originally delivered as a talk at Drupal Camp Asheville and Design4Drupal Boston.

Omega, a non-profit, mission-driven educational retreat center, with its primary campus in Rhinebeck, NY, sought the expertise of Savas to improve search on their website. 

Search is extremely important to them as they wanted to offer easier user engagement for the wide range of users that come to their site. They sought a search experience that would allow a user to see the breadth of information while also being able to easily drill into specific areas of interest.  Equally as important, Omega wished to “future proof” search development as they look forward to an eventual Drupal upgrade.

Previously, the search was much like a standard Drupal search. It had an Apache Solr back end and integrated with Drupal. It wasn’t very dynamic and lacked both speed and efficiency. 

Omega wanted to provide an effective way for visitors to quickly discover and reserve an experience through their website. In our research, we learned that the search page, though popular, had lagging conversion rates. The design and functionality were not taking advantage of the power of Apache Solr. Our goal was to allow visitors to hone in on their interests across topics, locations, schedules, and teachers. 

With a progressively decoupled approach, we replaced the Drupal search view with an embedded React application that has an app-like feel, interfacing with the speedy Apache Solr search index directly while also accessing Drupal’s robust and editable metadata for each piece of content.

This is how we did it. 

First, let’s get on the same page with terminology. 

Traditional monolithic Drupal handles the full request and response by rendering and delivering the entire page. 

Drupal is a data provider that consumers can access. Decoupled Drupal is a strategy where Drupal is used strictly as a content management tool, without a presentation layer, where you have Drupal as a central data store. This is sometimes referred to as “headless Drupal”. 

The options are limitless, but the essential point is that Drupal is the data store and different end devices are accessing that datastore and choosing how to present it (i.e. Amazon Echo will present it as voice). 

In Progressively Decoupled Drupal a JavaScript framework is layered on top of the preexisting Drupal front end. It allows Drupal to do the heavy lifting and render most of the page while the front end framework renders discrete parts of the page. 

The React app communicates with Drupal endpoints and updates parts of the page dynamically without refreshing the entire page. 

Often there is a lot of confusion around Drupal Search and the different options that are available.

Core search 

Pros: Comes out of the box with Drupal, stores the search index in the DB, has a simple configuration

Cons: Doesn't scale well

Search API module provides a framework for easily creating searches on any entity known to Drupal, using any kind of search engine. 

Search API Solr search provides a Solr backend for the Search API module. The backend uses Apache Solr servers for indexing and for searching content. It has great performance, is suitable even for use on large commercial websites and supports facets and multi-index searches.

When you introduce the browser, it makes the request and is then sent the rendered page. Essentially we are rendering the entire page in Drupal, but there are different components within that page that are making asynchronous requests to other resources, in some cases Drupal and in some cases Apache Solr. 

So, in the previous Omega Search when a user clicked “next” to see additional workshops, an entirely new page was returned by Drupal. This old search is slow, there is no immediate refresh or “show more”. 

With the Progressively Decoupled Architecture, Drupal is only pushing data to Solr. Note here that the box around Drupal is the project root on the server. Drupal is returning essentially a blank page with the javascript App.js. The proxy.php script is a lightweight PHP script that It provides some basic security and does not bootstrap Drupal, except in the case of autocomplete as explained later.

<div class="mail-container">

   <a id="main-content"></a>

      <noscript>
              <div class="region region-content">
                     <div class-"container">
                            <div class="message error">
                                   <p>Sorry, you must enable JavaScript to use the search page.<p>
                             </div>
                      </div>
               </div>
        </noscript>
    <div id="js-solr-serach" class="main-content solr-search containter"></div>

</div>

/**
 * Implements hook_preprocess_html( ).
 */
function omega_solr_preprocess_html(&$variables) {
   $current_path = current_path( ) ;
   if ($current_panth === 'search' ) {

     drupal_add_js( [
         'omegaSolr'  =>  [
             ' topics '  => _omega_solr_get_topics_list( ),
             ' teachers '  => _omega_solr_get_teachers_list( ), 
             ' keyInitiatives ' => _omega_solr_get_key_initiatives_list( ), 
             ' userIsPaidMember ' => _omega_membership_is_paid_member( ),
          ] ,
      ] , options: ' setting' ) ;

      // Reference your own stylesheet.
      drupal_add_js(
            data: Drupal_get_path ( type: ' module ' , name: ' omega_solr ' ) . '/js/dist/omega_solr_app.js ' ,
            ['weight' => 1, 'scope' => 'footer']
      ) ;
   }
}

By adding the JS via drupal_add_js() we can also pass data to the app. For example user role (paid member), lists of taxonomy terms (topics, key initiatives) and nodes (teachers) to be used in pre-loading filters.

Lastly, there is autocomplete integration. While Solr suggester was an option for auto-completion, it was more complex to implement and likely less manageable by the client. Proxy.php checks for autocomplete requests and handles them in a unique way.

In the end, we delivered a best-of-both-worlds experience for blazing-fast interaction with the ability for richer features and website admin control.