Solving the Problem of 'Nobody Reads the Manual…'! How to Use Search Logs

"I created and published a manual, but not many people are reading it…"

This is one of the most common concerns we hear from our clients.

You put in all that effort, only for it to go unread… It's truly disheartening. Let's put that frustration behind you.

Analyzing search logs can give you hints about what manuals are needed

When we say "people aren't reading the manuals," there can be many different reasons behind it.

One of the most common issues is:

"What they need isn't there (they can't find the information they're looking for)"

For example, consider this kind of user behavior:

Something came up that they didn't understand, so they tried looking at the manual.
There was no manual covering what they wanted to know...
They searched as best they could but couldn't find the information they needed, and felt completely let down.
The problem went unsolved. They looked up at the ceiling and silently vowed never to come back here again...

It may sound a bit dramatic, but it's entirely plausible.

(As a side note, we also often hear the comment that the manuals that have been created only cover things people already know, so there's no need to read them.
However, those manuals are valuable assets that exist to be viewed by the right people at the right time —
such as new employees handling a task for the first time, or even experienced staff tackling something they haven't done in a while.)

To avoid such situations as much as possible, maintaining manuals that people actually need is extremely important.

That's where the Teachme Biz feature called "Search Logs" comes in handy.

By learning about users' pain points through the "Search Logs," you can get a clearer idea of what kinds of manuals are needed.

This makes it easier to identify which manuals to create next and take action accordingly.

What are Search Logs?

When you search for something on the internet, you type in words into Google or Yahoo! JAPAN, right?

Teachme Biz also has a feature that allows users to search for manuals.

The "Search Logs" feature lets you view a report (list) of the "search terms" that were used in those searches.

Where can you access it? (Who can access it? What time period does it cover?)

First, log in to the web browser version of Teachme Biz. You can access the logs from the "Reports" menu at the bottom of the left-hand menu.

You can view either:

1. Search data for the entire group
2. Search data for a specific folder

1. For group-wide search data: accessible by Group administrators and sub-administrators
2. For folder-specific search data: accessible by Folder administrators and sub-administrators

The data can be reviewed going back up to a maximum of one month, so it is recommended to check it regularly each month, such as at the beginning or end of the month.

The data can be exported as a CSV file, which is recommended when conducting more detailed analysis.

What information can you see?

First, you can see the "Search Terms" — the actual words that were typed into the search box.

You can also see the "Type," which classifies results into the following three categories based on whether a manual containing the search term exists and whether it was viewed:

In addition to the above, you can also view information such as "Device" (the device or application type used during the search), "Manual ID" (a 7-digit number assigned to each manual), manual title, folder name, access time (yyyy-mm-dd hh:mm:ss), and number of hits.

What the data analysis reveals

From the "Search Terms," you can infer what users were struggling with (what they wanted to know).

From the "Type," you can see whether a manual existed for that search term, and whether the user actually went on to view it or not.

By categorizing and counting this data, two key facts emerge:

1. Trends in what users are struggling with
2. Whether the necessary manuals to resolve those issues are in place

Example workflow for analyzing search log data

For example, you might read through the data in the following way.

Looking at the "Search Terms," you can see multiple search terms related to "folders," which suggests that users were experiencing issues related to "folders."

Additionally, from the "Access Time," you can see that similar searches were conducted in succession.

This suggests that the same person was repeatedly searching because they couldn't access a folder and were having trouble.

Looking at the "Type" for those searches, you can see many entries of "No Search Hit" and "Search Hit - Not Viewed."

Following the timeline, searches like "folder can't view" and "folder unable to see" returned "No Search Hit," so the user tried several different keywords.

At some point, a search did return a hit, but the user did not go on to view any manual.

This is likely because the manual they were looking for wasn't visible among the results, or because the hit count was high at 19 results, prompting them to add more keywords to narrow down the search.

Narrowed-down searches such as "guest folder can't view" returned "No Search Hit," but ultimately the search "guest folder" resulted in a "Search Hit," and it is clear that the user viewed the manual titled "Add/Remove Members and Guests to a Folder."

In other words, the user's issue was resolved.

"And all was well!"

— but we can't quite say that just yet. If the analysis ends there, it has little value.

After analyzing the data, it is important to go further and take action to make improvements.

Improvement actions to take after data analysis

Perspective 1: No Search Hit (= No matching manual found)

This is a situation where no manual containing the search term existed.

The issues at play are the following two points:

1. No suitable manual exists for the user's problem
2. A suitable manual actually exists, but it did not contain the search term

As a follow-up improvement, if no suitable manual exists for the problem, take action to create one.

On the other hand, as illustrated in the "folder" search example above, there are also cases where a suitable manual actually exists but was not returned in the search results because it did not contain the search term.

In such cases, it is recommended to make the manual discoverable by:

• Adding the term to the title
• Adding the term to the description
• Adding it as a search tag

In particular, for searches that can be reasonably anticipated — such as "guest folder can't view" or "folder unable to see" — using search tags is a good approach.

Perspective 2: Search Hit - Not Viewed (= Manual found but not viewed)

This is a situation where a manual containing the search term existed, but the individual manual was not viewed.

It can be inferred that while manuals appeared in the search results, the user determined from the manual title or description that those manuals would not solve their problem.

The issues at play are the following three points:

1. A suitable manual actually existed, but it could not be identified by its title
   (it did appear in the actual search results)
2. A suitable manual actually existed, but it did not contain the search term (it did not actually appear in the search results)
3. No suitable manual existed for the user's problem

Which of A through C applies can be inferred to some extent by considering the work context related to the "Search Term."

For example, a search term like "folder can't view" suggests that the user needs a manual about "not being able to see a folder that should be there," making it clear what kind of manual should be provided.

If you cannot infer the answer from the "Search Term," interview the users and gather their feedback.

Representative examples of points to gather feedback on are as follows:

• Did the search term return the information the user was looking for?
• Was the title named in a way that was easy to find?
• Was the content of the manual easy to understand?
• Was the user actually able to resolve their issue? (i.e., are the steps reproducible?)

Based on the findings from your data and interviews, take the following actions:

• Create a new manual
• Revise the steps of an existing manual to make them clearer
• Update the title of an existing manual
• Add search tags

Perspective 3: Search Hit (= Manual found and viewed)

This is a situation where a manual containing the search term existed and was actually viewed.

This tells you that at least one manual that seemed promising — "maybe this will solve it…" — appeared in the search results and was actually viewed.

However, whether the manual actually resolved the user's issue cannot be confirmed from the report alone.

If, when cross-referencing with the "Search Term," there is a possibility that the viewed manual may not have resolved the issue, it is a good idea to interview users and gather feedback, as with Perspective 2.

You can also enable Teachme Biz features such as the "Manual Rating Feature" or "Comment Feature" to collect feedback directly within Teachme Biz on whether the manual was actually helpful.

"Teachme Biz saved me when I was stuck!" — That experience drives adoption

When a manual covering what a user needs simply doesn't exist, it becomes a source of frustration — all that searching and still no solution. This can lead users to feel that Teachme Biz has no value for them.

If this continues, even when users encounter a problem, they may think, "Searching Teachme Biz won't solve it, so I'll just ask someone…" — creating a barrier to actually using the tool.

To prevent this, it is necessary to understand user needs based on data and steadily create and improve the relevant manuals.

As is often said, a manual is not something you create once and leave — continuously improving and updating it is extremely important.

By repeating the following improvement cycle, user adoption will gradually take hold:

1. Identify specific pain points through data, interviews, and surveys
2. Request relevant departments to create or revise manuals
3. Actively communicate updates when new manuals are created or existing ones are revised

In this article, we introduced step 1 of the improvement cycle. We hope you found it helpful!

If you are responsible for creating manuals, please consider incorporating the improvement cycle into your regular workflow.