SEARCH INDUSTRY BENCHMARK STUDY Research conducted by Lucidworks—accelerating search relevance for the world’s most successful brands Critical Challenges Facing Today’s Search Practitioners A survey was conducted by Lucidworks to gauge the importance and difficulty of delivering relevant search results for customers and employees. All respondents rated search relevance as “Highly Important.” However, 96% of respondents found delivering relevant search to be a “Medium” or “High” difficulty task. The challenges of delivering relevant searches included too many stakeholders, a lot of signal data to collect, and other miscellaneous factors influencing relevance. Collecting quality signal data was considered the most important and also the most difficult aspect of delivering an effective search. Respondents ranked delivering more effective search relevance as the most important and delivering search relevance faster as the second most important of the three options. This indicates the overall desire of practitioners to fulfill their ultimate commission; to accelerate search relevance for users. 76% of respondents expect search relevance to become even more important in the future. The majority of respondents believe delivering relevant search will become more difficult. Practitioners wish for more perfect signals, accurate data, better analysis of user intent, more transparency of models, and help from AI and deep learning technology to accelerate relevance. Executive Summary 2 Critical Challenges Facing Today’s Search Practitioners © 2023 Lucidworks 3 Website Search and Browse Defines the Digital Experience Today’s consumers and employees utilize a company’s website search and browse experience as an integral part of their journey to learn or to purchase. Just like searching the internet using Google, Bing, or other internet search applications, getting relevant results is key to a great user experience. Today’s digital journey often spans engagements in mobile, desktop, tablet, and social media across websites, in-store experiences, and various other touchpoints. When stitched together the right way, all of those touchpoints (including the behavior-intent data captured from users) can be utilized to greatly improve the relevance of a customer or employee search. Lucidworks, a provider of technology and solutions that accelerate search relevance for customers and employees, conducted research to better understand what search practitioners think about the topic of search relevance. How important is delivering search relevance, how difficult is it, what obstacles are in the way, and how could the process be easier? For this study, search relevance is defined as the measure of how closely and accurately the generated search results relate to the user’s search query, and more specifically, the user’s true intent. A high search relevance improves a customer or employee experience, and a low search relevance has the opposite effect. Critical Challenges Facing Today’s Search Practitioners 4 Lucidworks fielded a survey in January 2023 to over fifty thousand search practitioners. Survey invitees were in North America, Europe, and Asia Pacific. Respondents were qualified first by self-selecting if they were directly involved in their company’s search activities. Respondents were in a functional role that was either technical in nature, such as search engineer, data scientist, or merchandiser, or in a business-focused role, such as digital commerce, marketing, operations, or knowledge management. Respondents were mid-to-senior level practitioners with manager, director, VP, or C-suite-level titles. TARGET TITLES TARGET ROLES VP Manager Director Knowledge Management Operations Marketing Search Engineer Merchandiser Digital Commerce Data Scientist Research Methodology 4 Critical Challenges Facing Today’s Search Practitioners © 2023 Lucidworks C-Suite The initial question was about the importance of delivering search relevance for customer and employee searches. Without question, practitioners feel strongly about the need for relevance, with 100 percent of respondents ranking search relevance as Highly Important. When asked about the difficulty level of delivering relevant search, 96% of respondents indicated it was medium or high difficulty. Only 4% said it was low difficulty. For programs at mid-to-large companies, these responses are logical. How important is delivering relevant search results? High importance 100% Medium importance 0% Low importance 0% How difficult is delivering relevant search results? High difficulty 56% Medium difficulty 40% Low difficulty 4% Complexity is inherent due to the data and signals required, the analytics needed, and the actions and triggers that occur to support large commerce and employee search programs. Search Isn’t for the Faint of Heart 5 100 % ranked search relevance as highly important, while 96% said it is difficult to deliver Lucidworks conversations with customers mimic the results from the survey. Search practitioners understand how critical search is, but struggle to find the resources and technology to enhance the search experience. Critical Challenges Facing Today’s Search Practitioners “Difficult to show relevant results. Difficult to autocorrect what the user meant to type and guide them to the right place difficult to show related results” “Different views on what is relevant. Different ways of expressing search terms” Describe your biggest challenge to delivering highly-relevant search. “Content quality, different data types, crawl and configuration of non-required metadata (noise), and user’s context/intent play major roles” “Having relevant, specific, and consistent product data. Understanding how context and intent influence relevancy at a given time in the discovery journey” 6 © 2023 Lucidworks When asked about the challenges of delivering highly relevant search, respondents said it was difficult because there were too many stakeholders involved, a lot of signal data to collect and sort, and the many factors influencing relevance. “All the filters and signal data that are required to narrow the search criteria to result in positive outcomes” “Too many stakeholders to make necessary changes” To deliver an effective search, respondents were asked to rank in order of importance three options: Collecting quality signal data about search relevance Analyzing search relevance-related data using analytics reports Taking correct actions (tuning, precisions, ranking) from search relevance-related data. Collecting quality signal data about search relevance was ranked most important. This signifies the need of having the right—and accurate—data flowing into a search platform. Practitioners understand that the basic ingredients of relevance are the right signal data collected through beacons, data streams, customer data platforms, and behavior tracking. Analyzing relevance data using analytics ranked second. Modern reporting encompasses a view of omnichannel signal data brought together with channel and intent analytics for practitioners to better understand what story the data is telling, then what actions must be taken. To deliver effective search relevance, what is most important? Collecting quality signal data 36% Analyzing data using analytics reports 34% Taking correct actions (tuning; recall, precision, ranking) from search relevance-related data 30% 7 Search Complexity is Daunting “Today’s users provide a wealth of data and signals from clicks, purchases, and implicit inputs from customer data platform information. The opportunity to deliver a highly relevant search experience, in real-time, is realistic. Companies that do this the best will drive optimal results from their search management efforts.” – Lana Klestoff, Lucidworks Chief Customer Officer 1 2 3 Collecting quality data was the highest ranked criteria for delivering effective search Critical Challenges Facing Today’s Search Practitioners 8 Respondents were then asked to rank the previous three options in order of difficulty. But unlike the previous answer, respondents now show that collecting quality signal data and taking corrective actions from search data are the most difficult. Today’s omnichannel environment means multiple data streams need to be connected, tested and continually monitored to provide batched or real-time data flows. This often requires IT/engineering, permissions and workflow discussions, security reviews, vendor management, and partner or outside consultant help. Taking correct actions from search data was also ranked as difficult. This signals the burden of tuning models to take action confidently, and in attaining the precision needed for next-step recommendations for team members or vendors. 8 Critical Challenges Facing Today’s Search Practitioners © 2023 Lucidworks What is the hardest part of delivering relevant search results? Collecting quality signal data about search relevance 37% Analyzing search relevance-related data using analytics reports 27% Taking correct actions from search-related data 36% “The key to a great digital experience for the user starts with searching and browsing. But the effort, processes, and requirements to power that simple experience can be daunting to manage. Fortunately, technology and the right expertise can help to prioritize and execute what is needed for success.” –Mike Sinoway, Lucidworks Chief Executive Officer It’s About Signals and Action Respondents were asked to select one answer from three options regarding the importance of relevant searches in the future. 76% of respondents said search relevance will be More Important in the future. This response may stem from growing consumer and employee expectations that digital experiences be faster, more connected, and easier to navigate. At its core, delivering relevant searches should be all of these things. In the future, you anticipate relevant search to be... More important to deliver 76% The same importance to deliver 24% Less important to deliver 0% In the future, you anticipate relevant search to require... More effort to deliver 40% Less effort to deliver 32% The same effort to deliver 28% The complexity of understanding how to deliver relevant search and all that is needed to make it happen will not readily get simpler. Respondents’ views are that data collection, analysis, and acting on that data will grow in complexity as search environments and signals grow. But for some, the use of vendor partners, data connectors, pre-trained models, expert data scientists, and even artificial intelligence and machine learning will help fulfill the promise of simplicity in delivering search relevance. 9 The Future of Search Looks Bright Regarding the difficulty of delivering relevant searches in the future, 40% of respondents indicated delivering relevant search would take more effort, 32% of respondents indicated it would take less effort, and 28% of respondents indicated it would take the same amount of effort. 76 % view relevant search delivery as more important in the future Critical Challenges Facing Today’s Search Practitioners © 2023 Lucidworks “The perfect set of signals including how to deal with new products, upcoming promotions, the cadence of local stock levels, etc., to remove the last areas that require merchandiser input” “More analysis on what users want; intent” If you had a magic wand, what would you fix in the process of delivering relevant search results? “Accurate product data targeted at delivering relevant search results combined with an understanding of query context and customer intent” “Smart insights regarding all data that is indexed. What search results are popular and why? What search results are not popular and why? Trends or changes in customer behavior, with relevance?” 10 When asked what they would fix about the process of delivering relevant search results if they could wave a magic wand, respondents stated their personal wishes: the need for more perfect signals, accuracy in data—including product data, better analysis of user intent, more transparency of models, less time spent fine-tuning results and models, and help from AI and ML. “Dynamic refiners to match with search results” “Reduce the amount of time spent fine-tuning results; less manual and more AI/automated” “Having more visibility into what weight to give all KPIs; the algorithm” “AI to assist in the quality collection of relevant signal data” Respondents were asked to rank the importance of three options: Delivering search relevance faster Delivering search relevance more easily for teams Delivering more effective search relevance for users. Respondents ranked delivering more effective search relevance as the most important and delivering search relevance faster as the second most important. These may indicate the overall desire of practitioners to fulfill their ultimate commission; effective search in an accelerated way. What is most important? Delivering more effective search relevance for users 42% Delivering search relevance faster 31% Delivering search relevance more easily 27% The importance they assign to effective (or accurate) relevance indicates their understanding of their goal. They can deliver search, but without accurate relevance, they miss the mark. In parallel, if they deliver relevant searches, but too slowly, the moment is lost and the experience is not optimized. The goal is to deliver relevant results quickly. Effective teams understand that using the best signals and data, combined with effective models and analytics, and actions provide a seamless and effective search experience in consumer and employee environments. Ultimately, a total, connected experience is what companies must deliver. 11 “ Clearly, the priority of respondents is to increase the delivery of search relevance. But that is difficult for many reasons. The promise of new and better technology—like AI and ML— combined with impactful models and real-time data offers the ability to deliver a better search and browse experience in an accelerated way.” –Peter Curran, Lucidworks Chief Revenue Officer Accelerated Search Relevance is the Goal 1 2 3 Improve search relevance using better technology, models, and actions Critical Challenges Facing Today’s Search Practitioners 12 To improve search relevance, respondents were asked to rank the importance of three options: Different or better technology Different or better expert guidance/consulting Different or better practitioner professional development Respondents indicated that having different or better technology was most important to improve search relevance. In this research, when asked to ‘wave a magic wand’ and fix the difficulties of delivering relevant search, respondents indicated several methods, but a primary one was the use of more artificial intelligence and machine learning. Despite today’s highly advanced search technology, it’s possible that practitioners feel these solutions are still too complex to implement. Respondents selected having different or better expert guidance/ consulting as the second choice for improving search relevance. More and more, search leaders in mid-to-large businesses are outsourcing help that will complement their own internal teams, allowing them to keep team size smaller and more agile, with less cost burden. Expert external teams that understand not only search relevance, but also the sector/vertical knowledge that companies need is becoming more important. Relevance is highly contextual and personal, and solutions providers need to understand user’s unique needs and intent. Respondents selected different or better professional development last in importance to improve search results. This is still important to respondents, but not as important as technology and consulting. Practitioners actively participate in associations, communities, training and education, events, and thought leadership consumptions to stay up on the latest trends and best practices. 12 © 2023 Lucidworks To improve search relevance, what is needed the most? Different or better technology 39% Different or better expert guidance/consulting 33% Different or better professional development 28% 1 2 3 Critical Challenges Facing Today’s Search Practitioners Respondents were asked to rank how useful prebuilt or out-of-the-box solutions are for search management. 40% answered they were very or extremely useful, while nearly 40% said they were somewhat useful. 17% said they were Not useful. Respondents like prebuilt models as a starting point because they jumpstart the process, but they still require tuning and updating. There is not an ‘easy’ button with search intent models, but practitioners want to take advantage of anything that resembles a quick start. 13 Mixed Results for Out-of-the-Box Solutions How useful are prebuilt/ out-of-the-box solutions for your search management efforts? Extremely useful 22% Very useful 22% Somewhat useful 38% Not so useful 18% Not at all useful 0% There is not an ‘Easy Button’ for quality search even if practitioners may hope for one Critical Challenges Facing Today’s Search Practitioners “Search is complex enough but new technologies such as ChatGPT, Bard, etc., are setting new expectations about how computers should react to human interactions, which in turn creates more complex queries to process. Being able to adopt these technologies within a flexible framework, and respond with minimal human oversight, is crucial to future-proofing search investments.” –Nick Jones, Lucidworks Managing Director Rest of World How important will AI be in the future for delivering relevant search results? Extremely important 52% Very important 36% Somewhat important 8% Not as important 4% Not at all important 0% Respondents believe artificial intelligence (AI)—including machine learning (ML)—is important to delivering relevant searches in the future. 88% of respondents claim AI is very or extremely important in the future of search. Artificial intelligence (AI) has already revolutionized the field of search relevance by providing algorithms that can interpret the intent of a user’s query in a more human-like manner. With the growth of big data, AI has become increasingly adept at analyzing large amounts of data to provide relevant results. This has led to a significant improvement in the accuracy and effectiveness of search results. One of the key areas where AI excels in search relevance is natural language processing (NLP). NLP is a branch of AI that deals with the interaction between computers and humans through language. AI algorithms can analyze the structure and context of a user’s query to provide relevant results that match their intent. 14 Critical Challenges Facing Today’s Search Practitioners © 2023 Lucidworks 88% Believe AI Will Be Important AI also excels in search relevance through personalization. AI algorithms use information such as a user’s search history, location, and preferences to personalize search results and provide a more tailored experience. This is achieved through machine learning algorithms that can identify patterns and relationships in data and use this information to improve search results. For instance, Lucidworks uses AI algorithms to recommend products based on a user’s past purchases and searches. In addition, AI can enhance search relevance by identifying and eliminating low-quality results. AI algorithms can analyze the quality and relevance of content to determine if it should be included in search results. This helps to improve the overall quality of search results and ensure that users are only presented with relevant and trustworthy information. “AI will not solve all search issues, but harnessing what it does best—complex processes at scale—is a natural application to the immense data sets and models that help make search and browse more relevant.” –Melissa Hill, Lucidworks, VP Product Management Accelerate Relevance Summary of Results Delivering search relevance is critical but difficult. Practitioners are looking to artificial intelligence and other new technologies to help. Search relevance will be more important in the future. Accelerating search relevance is the highest priority for practitioners. Accelerating search relevance is the highest priority for practitioners. Accelerating effective search relevance is crucial in today’s customer and employee shopping and informational search experience. In a highly competitive and fast-paced market, delivering accurate and relevant search results greatly impacts customer satisfaction and loyalty, and impacts employee efficiency and productivity. Poor search relevance leads to frustration, decreased trust, and even loss of business. On the other hand, providing relevant and user-friendly search results can greatly improve the overall user experience, increase customer engagement, and drive conversions. In this research, respondents ranked delivering more effective search relevance as the most important and delivering search relevance faster as the second most important objective. This indicates the overall desire of practitioners to fulfill their ultimate commission; to deliver effective search in an accelerated way. Get Started Lucidworks believes that the core to a great digital experience starts with searching and browsing. We provide search technology and solutions to accelerate relevance for the world’s most innovative organizations. Learn more at Lucidworks.com © 2023 Lucidworks 1 2 3 4