Solving the Problem of 'Nobody Reads the Manual…'! How to Use Search Logs

"I created and published a manual, but not many people are looking at it..."

This is one of the most common concerns we hear from our clients.

You put in all that effort, only for it to go unread... it's truly disheartening. But let's put that feeling behind us.

Analyzing search logs can give you hints about what manuals are needed

When we say "people aren't reading the manuals," the reasons can vary widely.

One of the most common reasons is:

"The manual they need doesn't exist (they can't find the information they're looking for)"

For example, consider this kind of user behavior:

Something came up that they didn't understand, so they tried looking at the manual.
But there was no manual covering what they needed to know...
They searched hard but couldn't find the information they wanted, and walked away disappointed.
The problem went unsolved. They looked up at the sky and swore they'd never come back...

A bit dramatic, perhaps, but it's entirely plausible.

(As an aside, we often hear that the manuals that do exist only cover things people already know, making them unnecessary to read.
However, those manuals are valuable assets that exist to be read at the right time by the right people —
such as new employees tackling a task for the first time, or even veterans returning to a task after a long break.)

In order to avoid such situations as much as possible, keeping the manuals that are actually needed in order is extremely important.

That's where the "Search Log" feature of Teachme Biz comes in handy.

By understanding user pain points through the "Search Log," you can get a good sense of what kinds of manuals are in demand.

This helps you identify what manuals to create next and makes it easier to take action.

What is the Search Log?

When you search for something on the internet, you type words into Google or Yahoo! JAPAN, right?

Teachme Biz has a similar feature that lets you search for manuals.

The "Search Log" is a report (list) that lets you review the "search terms" that were entered into that search bar.

Where can you access it? (Who can access it? What time period does it cover?)

First, log in to the web browser version of Teachme Biz. You can access the log from the "Report" menu at the bottom of the left-hand menu.

You can view either:

1. Search data for the entire group
2. Search data for a specific folder

The access permissions are as follows:

1. For group-wide search data: Group administrators and sub-administrators
2. For search data by specific folder: Folder administrators and sub-administrators

Since data can only be retrieved for a maximum of one month, it is recommended that you check it regularly each month, such as at the beginning or end of the month.

You can export the data as a CSV file, which is recommended for more detailed analysis.

What information can you see?

First, you can see the "Search Terms" — the actual words that were typed into the search bar.

You can also see the "Type." The type classifies whether a manual containing the search term exists and whether it was viewed, into the following three categories:

In addition to the above, you can also view "Device" information (the device or application type used when searching), "Manual ID" (a 7-digit number assigned to each manual), manual title, folder name, access time (yyyy-mm-dd hh:mm:ss), number of hits, and more.

What can you learn from data analysis?

From the "Search Terms," you can infer users' pain points — what they were trying to find out.

From the "Type," you can see whether manuals existed for a given search term, and whether users actually went on to view those manuals or not.

By categorizing and counting this data, two key insights emerge:

1. Trends in user pain points
2. Whether the necessary manuals are in place to resolve those pain points

Example of a search log data analysis workflow

Here is an example of how you might go about reading the data.

Looking at the "Search Terms," you can see multiple terms related to "folders," which suggests that users were having trouble with something related to "folders."

Additionally, from the "Access Time," you can see that similar searches were performed in succession.

This suggests that the same person was repeatedly searching because they couldn't access a folder and didn't know what to do.

Looking at the "Type" for those searches, you can see many entries of "No Search Hit" or "Search Hit - Not Viewed."

Tracing the timeline, searches such as "folder cannot view" and "folder can't see" returned "No Search Hit," so the user tried several different keywords.

At one point, a search did return a hit, but the user did not go on to view the manual.

This is likely because the manual they needed wasn't apparent, or because the hit count was high at 19 results, prompting them to add more keywords to narrow down the search.

A narrowed search such as "folder can't see" returned "No Search Hit," but ultimately the search "folder add" resulted in a "Search Hit," and the user can be seen viewing the manual titled "Adding a Member to a Folder."

In other words, the user's problem was resolved.

"And they all lived happily ever after!"

— well, not quite. If you stop there, the analysis serves no purpose.

After analyzing the data, it is crucial to take improvement actions as well.

Improvement actions to take after data analysis

Point 1: No Search Hit

This is the situation where no manual containing the search term was found.

The issues at play are the following two:

1. No suitable manual exists for the user's issue
2. A suitable manual actually exists, but the search term was not included in it

As an improvement measure, if no suitable manual exists for the issue, take action by creating one.

On the other hand, as illustrated in the "folder" search example above, there may be cases where a suitable manual does exist but the search term wasn't included in it, causing it not to appear in search results.

In that case, it is recommended that you make the manual searchable by:

• Adding the term to the title
• Adding the term to the description
• Adding the term as a search tag

In particular, for searches that can be anticipated to some degree — such as "folder cannot view" or "folder can't see" — it is a good idea to make use of search tags.

Point 2: Search Hit - Not Viewed (= Search hit but manual not viewed)

This is the situation where a manual containing the search term existed, but the individual manual was not viewed.

The manual appeared in the search results, but it is assumed that the user determined from the manual title or description that it would not solve their problem.

The issues at play are the following three:

　　A. A suitable manual exists, but could not be identified from the title
　　(It did actually appear in the search results)

　　B. A suitable manual exists, but the search term was not included in it (it did not actually appear in the search)

　　C. No suitable manual existed for the issue

Whether A, B, or C applies can be inferred to a reasonable degree by considering the type of work related to the "Search Term."

For example, a search term like "folder can't see" suggests that the user is having trouble because "a folder that should exist isn't visible to them," which helps clarify what manual should be provided.

If you cannot determine the cause from the "Search Term" alone, conduct a user interview to gather feedback.

The following are representative examples of perspectives for receiving feedback.

• Whether the desired information was included for the search term
• Whether the title was easy to find
• Whether the content of the manual was easy to understand
• Whether the issue was actually resolved (i.e., whether it was reproducible)

Based on the results gathered from data and interviews, take the following actions.

• Create a new manual
• Revise the steps in existing manuals to make them easier to understand
• Revise the titles of existing manuals
• Add search tags

Perspective ③: Search Hit (= Search hit present, manual viewed)

This is a situation where a manual containing the search term exists and was actually viewed.

This tells us that at least one manual that users felt "might solve this..." appeared in the search results and was actually viewed.

However, whether the manual actually resolved the issue cannot be confirmed from the report alone.

If, when cross-referencing the "search terms," there is a possibility that the viewed manuals may not resolve the issue, it would be a good idea to interview users and gather feedback, as with "Perspective ②."

You can also enable features such as Teachme Biz's "Manual Evaluation Feature" and "Comment Feature" to receive feedback directly within Teachme Biz on whether the manual was actually helpful.

The experience of "Teachme Biz saved me when I was in trouble!" leads to lasting adoption

From the user's perspective, not having a manual related to what they want to know causes frustration when they searched hard but couldn't find a solution. This can also lead to users feeling that Teachme Biz has no value to them.

If this situation continues, even when users encounter problems, obstacles to tool adoption arise, such as thinking "Searching Teachme Biz won't solve it, so I'll just ask someone..."

To prevent this, it is necessary to understand user needs based on data and steadily create and improve related manuals.

As is often said, manuals are not finished once created — it is extremely important to continuously improve and revise them.

By repeating the following improvement cycle, users will become more consistently engaged.

1. Identify specific pain points through data, interviews, and surveys
2. Request creation and revision from relevant departments
3. Actively communicate updates when new manuals are created or revised

This time, we introduced step 1 of the improvement cycle. We hope you found it helpful!

For those responsible for creating manuals, please try incorporating the improvement cycle into your regular workflow.