In this video you will learn how to use the Keyword search module in MailCom's expert.tools. MailCom expert.tools is a browser application that provides versatile business data research and enhancement tools. Serious business professionals, marketing, sales and purchasing managers use expert.tools to leverage their business success.
In this video you will learn how you can crawl websites and identify which of these contain business critical keywords using the expert.tools keyword search module. You will also learn how you can enhance data quality using website information. Also you will learn about the different search parameters of the keyword search module. And at the end of this video, I will show you a real keyword search use case to illustrate its capabilities.
As you can see here, I am currently in the main app window of expert.tools after I have already uploaded an address file. If you want to learn about loading your address file, you can watch our video on loading address files into expert.tools. You will find it in the description of this video.
First of all, I will show you where you can find the keyword search module. To open the keyword search module window, go to the Task menu that you will find at the top of the screen. It is here in the dark top menu bar. Click on Keyword search. The keyword search module should now appear just above the business address table.Let me now explain which elements and blocks you can find on the screen of the keyword search module. The keyword search block is divided into three parts: Task, Parameters and Status. I will start at the left side with the task block.
The Task block contains the task name and a short description. The small dollar icon next to the task name gives information about the task’s cost. Costs are typically separated into execution and line processing cost. At the time of recording this video you see that there is no cost for running the module but 1 credit will be consumed for processing each website. You will also find the green Start button, which starts the keyword search, and the Settings button further below.By using the Settings button you can select the input and output columns. That way you can tell expert.tools which column contains the website and which column shall receive the result of the keyword search. This is called the output column. You can also change the module settings which means that you can choose the crawler profile to search not only the main page, but also sub pages, switch data synchronization to disc on or off, etc.
The second part of the keyword search block is the Parameters area. Here you can enter your specific keywords either as simple strings or as a Javascript Regular Expression also called RegEx. Later in this video, I will explain what a RegEx is. Select “Simple string” or “RegEx” depending on which type of search you want to use.You also have the option to decide whether expert.tools should search for the keyword within the html code or only in the human readable plain text of the websites. To switch between these two search modes select the respective dropdown value here.
Next to the input field, as well as to the two dropdowns, you will find a round black “i”- symbol, which gives you a short description on the input elements as soon as you move your mouse pointer over it.
And the third and last part of the keyword search block is the Status area which gives you some important information such as the start and end time of the task, as well as its duration. It also includes information about the exact amount credits used and other task specific information.
After I introduced you to the keyword search module window, I want to come back to the different search options I mentioned earlier. Now I will explain to you the difference between plain text and html search. I will do that using a few slides which I prepared for you.
Using the right search method can in some cases be crucial to the success of a keyword search.
The plain text search is the most commonly used search method because it can be used to find human readable information on a website. For this type of search you can use every keyword that is part of the website’s readable text. So if we have, for example, a list of lawyers, corporate law, criminal law or family law can be possible keywords.
The html search is a bit more special. Here, expert.tools scans the html code of the website. So, the html search is very useful if you want to search for page meta information and structural elements of a website, such as html-tags, descriptions and meta-tags. As keywords for the html search you can use, for example, “wordpress” to find out if the given websites were built with the Wordpress Content-Management-System or not. You also have the option to search for html-tags, as I already mentioned before. For example, these “lang=” tags to find out the website’s language.
While with a plain text search expert.tools checks every single word that is visible on the web page, with an html search expert.tools checks the entire html code of the web page. This means that not only the visible words are searched, but also, for example, html tags that are not displayed on the website.
To illustrate the difference of both search methods, I have prepared a simple example web page for you. This is the html code of it.
Let's look, for example, at this certain piece of code which is text followed by a link (<a href="https://www.best-car-workshop.com">This is the link to our workshop and garage website</a>) and a short text below.So, if you decide to use the plain text search, expert.tools would only check the words that have a white color in this text editor. This would only be the text before the link, the link text as well as the text which follows the link.If you do an html search instead, expert.tools would examine every single character of this code block, so for example you could search for every website which links to this URL “https://www.best-car-workshop.com”.
Another good example for html search could be if you want to find out which companies use chatbots or chat tools in general on their websites. We can also find an example for that in our sample website code as you can see here.This is the code to embed the “Tawk.to” chat widget. If you want to find out which websites use that chat widget, you would search for the keyword “embed.tawk.to” using html search. The plain text search would only search the human readable text (in our source code this is the white text), so it would not find any results because the keyword “embed.tawk.to” is not part of the human readable information on the website.
As you can see, plain text search can be used if you want to extract some specific information about the company whereas the html search should be used if you want to know something about the company’s website like the website’s language, if it uses a certain type of shop or certain marketing tags which can be found in the html code.
After I have shown you the difference between plain text search and html search, it’s also very important for you to know how and in which cases to use the two different keyword options: String and RegEx.In a string search, expert.tools searches the web pages for a single keyword. So if you only want to search for a single word e.g. “workshop”, string search is sufficient for you.However, if you want to search for multiple words, word variations or patterns, you should use RegEx search e.g. “EU-(reimport|new.car))”.
RegEx stands for Regular Expressions and is a sequence of characters that specifies a search pattern. There are several different dialects of RegEx. Expert.tools uses JavaScript Regular Expressions.To learn more about regular expressions, there are many good sources on the internet, such as regexlearn.com that offers information about RegEx as well as a cheat sheet that gives you an overview over the most important commands. regexr.com is also a very good website. It provides a RegEx sandbox where you can create and test your regular expressions.
In this video, I will just give you some examples of how regular expressions could be used for a keyword search on expert.tools. Let’s say, we want expert.tools to search for the two keywords “sustainability” and “equality”. Then you could just write “sustainability|equality”. As you can probably already guess, the pipe character in the middle is understood as an OR.
But let me give you a more difficult example. Let us search for every variation and synonym of automotive companies. That would be the four keywords: automotive company, automotive manufacturer, car company and car manufacturer. So, a possible RegEx statement for this case could be, for example this one: automotive\s(company|manufacturer)|car\s(company|manufacturer)
Here are two new RegEx components: backslash “s” and the brackets. While the backslash “s” is the RegEx statement for a whitespace, the brackets stand for a group of keywords. In our case, it's the OR combination that I have already explained in the previous example.You can read this whole statement like this: automotive, whitespace company or automotive, whitespace manufacturer - that's the first part. And the second part: or car, whitespace company or car, whitespace manufacturer. Instead of this regular expression you would have to write four string searches: automotive company, automotive manufacturer, car company, car manufacturer
I will now show you examples for every combination of a keyword search type and search mode. There are four possible combinations: plain text string search, plain text RegEx search, html string search and html RegEx search. But before we look into these examples, I want to show you an overview slide regarding the search dimensions.
There are two main types of search: On the one hand, the plain text search and on the other hand the html search, with which you can scan the html source code.So, if you now want to search just for simple search patterns, that means for single keywords within the human readable website text or the html code, then use eith
er the plain text string search or the html string search. But if you want to search for more complex search patterns like several keywords and keyword variations, you have to use either the plain text RegEx search or the html RegEx search.
After I explained to you the difference of the four search dimensions, I will show you a practical example for each of these four dimensions.Let’s start with the plain text string search. A good example for that could be a list of websites of car dealerships that could be searched for the keyword ‘workshop’. In this case you would use expert.tools to search within the plain
text for every car dealership that also has a workshop to repair and examine cars. Therefore, the keyword search parameters block would look like this: workshop
A situation where plain text RegEx search would be useful could be when you
have a list of lawyers and law firm websites and you would want to find out on which matter they are focused. You could search for specific legal terms, such as “banking”, “capital markets law” or “IPOs”. The RegEx code line for this example could look as follows (banking|capital market|IPO)As I already explained, the pipe character between each of the search terms stands for an OR.
An html string search can be used if you want to find out whether companies use a certain content management system for their website. To do that, just type the keyword wordpress into the keyword field and select “html text” and “string” in the respective drop downs. Run Expert.tools now to search for the keyword “wordpress” in the html code of every website on your list.
Finally, let’s talk about an example for html RegEx search. Imagine you have a list of companies and want to find out which of these companies have a French website.You could use the following RegEx code to achieve this:(lang=”(fr|frz|french|français|française))This regex is for keywords which are embedded in an html tag. The pipe character between the language keywords stands for an OR, so it will match any of “fr”, “french”, etc.
Now you are familiar with all four search dimensions. After so much theory, we will now start a keyword search to demonstrate how it works. I will do a plain text string search like the first given example with the car dealerships and workshops. Therefore, I have already loaded a business address list of fifty car dealerships and what we now want to do is to search every one of these car dealerships for the keyword “workshop”.
Now let’s select String search and plain text as search options and enter the keyword “workshop”.Now, I click on the Start button. Before the first time you are running a keyword search task, the settings window will pop-up. Here, expert.tools asks you to define the input column for the module. This is the column which contains the URLs of the websites that shall be examined. Let’s select “website” as the column which contains the websites and move on to the Outputs. Here we decide in which column the keyword search results should be stored. We choose the option to Always create a new field. This means a new column gets created whenever we run the keyword search.
Finally, we click on the Task settings tab. Here we can select the crawler profile and also set some special settings like the synchronization of the table data to disc.If you want to know more about editing and creating custom crawler profiles, check out the video description. There you can find a link to a special video about crawler profiles. For our example, it should be enough to select the “Main page only“ crawler profile.That means that expert.tools will search for the keyword only on the main page of each website. In most cases this is sufficient as the important information is often found on the main page.We keep the other settings as they are. Now click on the Save button and the settings window closes. Back in the main window, click the Start button. The keyword search task should now start.
As you can see, the progress bar has disappeared now and within the Status field we ca
n read that this task consumed 86 credits while processing 50 websites. Of these websites 45 have been analyzed successfully whereas 5 websites could not be loaded and read. Reasons for this can be that the server where the website is stored is down, the website is gone, the website returns a 400 or 500 error page, the connection is broken or our crawler is blocked by a protection mechanism.From the results you can also see that expert.tools added a new column to our address table.
The red crosses stand for the word false and mean that our keyword “workshop” was not found on the website and the green check mark stands for the word “true” and means that the keyword was found on the respective website.If you only want to display the websites on which the keyword was found, just enter true as a filter in the column header or if you want to display the websites on which the keyword was not found, enter “false” as filter in the column header.
After the keyword search task is finished, you can either start another task or export the new address file.
We have now reached the end of this tutorial. In this video we explained how the keyword search task of expert.tools works and which search options you have. In the end of this video, we have seen a real world keyword search example together with the results in the table.
If you want to learn more about expert.tools you will find more videos on our channel or on our expert.tools website. If you have any discussion items, questions or suggestions please let us know by commenting on the video below. You can also contact sales or just send us an email.
