Menu
homeErrors

The new status is not of sufficient quality. Pages in search. Pages with little content

The devil is not as scary as he is painted
- Russian proverb

Sometimes it’s quite difficult to understand what search engines want from you, what exactly they understand by the meaning of a “page” Low quality"? Sometimes you look at a page and frankly don’t understand why Yandex or Google didn’t like it. In this article we will look at pages that search engines remove from searches, considering them to be of insufficient quality.

Low quality pages as perceived by search engines

On the Yandex blog, Elena Pershina gives the following definition of a low-quality page: “By the concept of not quality page We understand pages whose likelihood of being shown in search is almost zero. For obvious reasons, we don’t tell you how to identify such pages, but this is a clear signal that if you want to see this page in search, then you need to do something with the page.”

Let’s make a slight correction to the words of the Yandex representative: since algorithms sometimes fail, and in favor of sites, a page can rank in the TOP, and then disappear from the search results without a trace.

That is, low-quality pages are documents that have no value, do not bring new and useful information, do not provide a relevant answer to the user’s question from the point of view of search engines.

How to detect deleted low-quality pages

Yandex.Webmaster

The easiest way to find excluded pages is to use the Yandex.Webmaster tool.

Go to the “Indexing” section – “Pages in search”.

Go to the “Excluded Pages” tab and select the “Insufficient Quality” status.

You can, without moving from “Last changes”, select the status “Deleted: Insufficient quality”.

Google Search Console

Unlike Yandex, Google does not provide information about which pages were removed from searches due to quality. Even in the new beta version Google Search Console, unfortunately this function is missing.

The only signal from Google right now is “The URL you submitted is returning a false 404 error.” In such cases, you need to analyze the pages to make sure that they exist and are not deleted (and the server’s response is simply incorrect). You can read more about soft 404.

  1. Let's go to new version Google Search Console.
  2. IN side menu Find “Status” – “Indexing sent URLs”.

  1. Select the line “The URL you sent returned a false 404 error.”

  1. We analyze excluded pages.

Reasons for deleting pages

Having analyzed a large number of various sites and having identified a pattern among pages deleted due to low quality, we came to the following conclusions:
1) The Yandex algorithm is not thorough: instead of classifying the page as deleted, for example, due to duplication, it excludes it as low-quality.
2) Low-quality pages are more often found on commercial sites - online stores and aggregators than on informational ones (due to automatic generation).

Typology of low quality deleted pages

1. "Victims" incorrect operation algorithm

1.1. Duplicate pages

Low quality pages often include duplicate pages.

Such pages are quite easy to identify if nothing except the URL is unique.

1.2. Sorting pages, pagination and GET parameters

More often, Yandex excludes such pages as duplicates, but, as shown in the example below, they can be deleted for another reason.

Sorting pages:

Pagination pages:

GET parameters:
In this example, the GET parameters specify the user's region.

1.3. Secondary mirror

The site has moved to a secure protocol. For a long time Yandex robot didn't know what to do with old page on HTTP and then deleted it as low-quality, then added it back to the search. As a result, a few months later, he deleted the page as a non-main mirror.

1.4. Pages closed in the robots.txt file

The directives in the file are not direct instructions for search robots, but serve more as recommendations. Based on practice, the Yandex bot adheres more to the settings written in the file than Google. But not at this time. As in the previous example, after “tormenting” the page several times, he finally “condescended” and removed it from the search as prohibited in the robots.txt file.

2. Really insufficient quality pages

This group of “I’m bad, useless and no one needs me” includes low quality pages that really are low quality.

2.1. Template generation of pages
Often template filling and generation of pages leads to errors internal optimization: non-unique Title, missing Description, something wrong with H1, etc.

It is important to say here that the user will easily understand the difference in the pages and they may even be useful for him, but he will not know about them, since the robots are guarding order and will not allow these pages to be included in the search.

When I saw the Title on the page, I didn’t have to wonder for a long time what was wrong with it.

2.2. Poor completion of product cards

It’s easy to create a product card, but filling it with high-quality content, product descriptions, and images is not something everyone can do.

2.3. Listing without listing

  • It is unlikely that such a page will make it to the TOP;
  • Most likely, the bounce rate on such a page will be maximum.

Yandex itself will tell you about this, excluding the page from the search as being of insufficient quality.

2.4. Pages with little content

Despite Yandex's words of support that it is not the quantity of content that is important, but the quality, its absence is not a good idea.
For example, on this page, except for the template part and two images, there is nothing.

2.5. Pages not intended for search

This group includes pages robots.txt, Sitemap, RSS feeds.
In almost every project you can find a robots.txt file, removed from the search as a page of insufficient quality.

Don’t panic, the robot knows and remembers about it. The page simply won't display in search results- well, why is it needed there?
XML and HTML sitemaps are also excluded. Especially if you have a multi-page sitemap - what is the use of it in search?

Yandex believes that RSS feeds have no place in search either.

2.6. Pages with incorrect server response

In this group we will include pages that should produce a 404 server response, but instead respond with 200 OK.

For example, these could be non-existent pages pagination. Yandex has excluded the eighth page of pagination, out of the existing seven.

It could also be blank pages. In this case, you need to analyze and make a decision: configure a 404 server response or a 301 redirect.

Google can remove pages like SOFT 404, but more on that later.

2.7. "Need to more pages»

Aggregators and online stores often sin with this, believing that the more pages, the greater the visibility and traffic. Pages are created without taking into account their quality and necessity.

For example, low-quality pages are those created for each question in the FAQ.

Often useless pages are created by applying all the filters. I agree that you need to think about visitors and the ease of use of the site. If an online store has a large assortment, then the site structure should consist of many categories and subcategories, and also have various filters. But, firstly, is it necessary to create a page for “20 cm porcelain dolls with blue eyes in a pink dress with a basket” and, secondly, are such pages needed in the search?

2.8. Technical errors

Yandex does not favor pages with a pop-up that overlaps the text without the ability to close it, or randomly created pages for images.

How it should be and how it should be implemented on the site:

Randomly generated page:

2.9. SOFT 404

As we already said, Google attaches just such a label to pages. These may be blank pages or pages with very little content.

Impact of low quality pages on rankings

No matter how many pages are excluded from the search for this reason, this will not affect the ranking of the remaining pages on the site.

This page removal algorithm analyzes each page individually, trying to answer the question: “If the page is searched, will it provide a relevant answer to the user’s question?”

As Yandex says, a page can be excluded from search even if there are no queries. The robot can return it if the number of requests to which the page is relevant increases.

What to do with low quality pages

You can take action only after you have determined the reason for excluding the page from the search. Without clarifying the circumstances, you should not panic, immediately delete pages, or set up 301 redirects.

Algorithm of actions after determining the reason for deleting a page:

Duplicate pages: 301 redirect or rel=“canonical”.

Sorting pages, pagination and GET parameters: set up rel=“canonical”/unique pagination pages.

Secondary mirror: we check 301 redirects and send them for reindexing.

Pages closed in the robots.txt file: if the page is not needed in search, set up the noindex meta tag.

Template generation of pages: If a page is needed in searches, we uniqueize it and work on quality.

Poor completion of product cards: add a product description, images, etc.

Listing without listing:

  • We check whether such pages brought traffic;
  • determine whether users need them;
  • whether the product is temporarily unavailable or whether it has never existed and will not exist.

We take actions based on the results. For example, if a page brings traffic and there is temporarily no product on it, you can display a feed with similar products or from related categories.

Pages with little content: We determine the need for such pages in searches; if they are needed, we fill them with quality content; not needed - set up the noindex meta tag.

Pages not intended for search: Everything is simple here - we don’t do anything, we just live with it.

Pages with incorrect server response and SOFT 404: No matter how logical it may sound, we set up the correct server response.

"Need more pages": we check whether such pages bring traffic, determine whether users need them in searches, whether queries are frequent - we take actions based on the result.

Pages from technical errors: we correct shortcomings / if the pages are not needed, close noindex / configure the 404 server response.

Conclusion

Unfortunately, search engine results are filled with junk, low-quality content, and meaningless generated pages. Yandex and Google are actively fighting such pages, excluding them from searches. We are for it quality content. Therefore, if you are having difficulties, you are watching how search engines are deleting pages citing insufficient quality, we can run your site and write instructions to solve the problem.

Regarding the unloading of indexed pages, Yandex finally did what was needed.

And now we have received very good tool with which you can get very interesting information.

Today I will tell you about this information and you can use it to promote your sites.

Go to Yandex.Webmaster, to the section "Indexing"

And here is this picture in front of you (click to enlarge):

This excluded page data gives us a lot of information.

Well, let's start with the Redirect:

Usually, a redirect does not pose any problems. This is the technical component of the site.

This is a regular page duplicate. I would not say that this is not so critical. It’s just that Yandex of two pages considered the second page more relevant.

And Yandex even wrote his comment: The page duplicates the page http://site/?p=390 already presented in the search. Provide the robot with your preferred address using a 301 redirect or the rel="canonical" attribute.

This can be used in the following way: Sometimes the pages of the site that you promote fly out and, on the contrary, their duplicates appear in the index. IN in this case you just need to register the canonical URL on both of these pages to the one you are promoting.

After that, add both of these pages to the “Robot Re-Crawl”.

This is the page in whose meta tags the canonical URL to the desired page is written.

Everything is fine here and this is the normal process of the site.

Yandex also writes a hint here: The page is indexed at the canonical address http://site/?p=1705, which was specified in the rel="canonical" attribute in source code. Correct or remove the canonical attribute if it is incorrect. The robot will track changes automatically.

This usually happens when you deleted some pages but did not install a redirect. Or the 404 error was not set.

This does not cause any harm to website promotion.

Well, this comes to the most interesting part. Insufficient quality page.

Those. the pages of our website were removed from the Yandex index because they were not of sufficient quality.

Of course, this is the most important signal to your site that you have global problems with these pages.

But not everything is as clear as it seems.

Often these are pages of pagination, search, or other garbage. And these pages are correctly excluded from the index.

But sometimes it happens that online store product cards are excluded from the index. And they exclude thousands. And this certainly indicates that there are some serious problems with your product card pages.

I've looked at many online stores this week and almost all of them have something similar. Moreover, page departures are observed in the tens of thousands.

There may be such problems that we have several identical pages where the product is just a different color. And Yandex believes that this is one page.

In this case, it’s either to make one page with a color choice on one page, or to modify other pages.

But of course it’s worth saying that this is a GLOBAL help for all online store owners. They made it clear to you which pages flew away and why.

Here you need to work on the quality of these pages. Maybe these pages duplicate others, maybe not.

Sometimes there is simply no text on such pages. And some have no price and Yandex removes such pages from the index.

I also noticed that if the product card page has the status “Product out of stock,” then such a page is also removed from the Yandex index.

In general, work.

About others interesting features I will tell you on Monday at my seminar -

Yes and more. Many people know this problem with Yandex.Advisor:

Those. you paid for a click from Direct, and Yandex.Advisor takes your paid client to Yandex.Market.

This is truly an outrageous case.

As I understand it, Yandex will not change anything.

Okay, then I'll change it myself.

This Yandex.Advisor primarily concerns online stores. And online stores are primarily based on engines: Bitrix, Joomla, Webasyst.

So for these engines I am writing an advisor blocker. Those. When you install this plugin on your engine, Yandex.Advisor will not work on your site.

I will later send these plugins to everyone who comes to my seminar for free.

I chose the most popular engines on which IM is installed. This is not necessary for service sites. But for IM, that’s it.

If you have questions, ask questions.

I think it’s no secret that some pages of a site sooner or later disappear from the search engine, there can be many reasons for this - not original content, when the page was crawled by a robot, it was inaccessible, all sorts of violations on the part of the webmaster, and so on.

However, there is another fairly common reason, which the Yandex Webmaster notifies us about - low quality page. What does a page of insufficient quality mean? Let's try to figure this out.

I have known the problem of excluding website pages from the index for a long time, but for me personally it was not critical for one simple reason - these pages were not very popular. And on this site similar problems was not noticed at all until recently:

The screenshot shows that the Yandex robot excludes the page from the search with a note that it is not of sufficient quality, and when next update search base the page is back in the index. Yandex explains this by saying that the exception occurs as a result of the work of a special algorithm, which for some reason considered this page not relevant to queries.

To be honest, this answer did not suit me and I wanted to know the specific reason for excluding a page from the index. After all, everything seems to be fine - the content is unique, the page has been in the index for quite a long time and there have been no problems, besides, no changes were made to this page and suddenly the page became of poor quality.

And then I decided to write to Yandex support to get specific answers to all the above questions. We must pay tribute to Yandex technical support, they responded really quickly. You can see a screenshot of the response to my appeal below, I’ll explain why the screenshot is later:

The answer is basically clear, special algorithm evaluates the page and makes a forecast of its display in search results. If the forecast is disappointing and the algorithm decides that users simply will not be able to find this page, then it is excluded until better times. In addition, Yandex technical support assures that this will not affect site traffic in any way. It seems that everything is clear and at the same time everything has become even more confusing.

But imagine my surprise when I checked this text for uniqueness. As you probably already guessed, the uniqueness was, frankly, zero, there was an incredible number of sites where this answer had already been published and, at a minimum, all these sites help improve the secret ingredient algorithm of Yandex. Actually, that’s why I took a screenshot of the letter, so as not to produce copies of it.

Having opened a couple of sites on which the letter with the answer was published why Yandex removes pages from search, I was even more surprised to see the publication date from 2014. It turns out that Yandex has been providing standard unsubscribes for at least 4 years. And no promised improvements were noticed.

Why Yandex removes pages from search

Having analyzed all those pages that from time to time Yandex considers to be of insufficient quality and excludes them from the search, I came to a completely logical conclusion - it’s all about behavioral factors. And personally, my vision is something like this:

  • If users find a page of your site in the search, but it does not meet their request, they close it immediately, this is the first big minus for you current page. Actually, this situation just speaks of the irrelevance of the page.
  • The second disadvantage that this or that page can receive arises from the fact that users who come to the site stubbornly do not want to go to certain pages, they see links to them, but don’t click.

Of course, all this is my personal guesswork, precise work I don’t think anyone knows the algorithm except its developers and employees of Yandex itself (and even then not all of them). But all these guesses seem quite logical to me personally.

After all, it’s easy to exclude from the search irrelevant page I think it’s unnecessary, the most that can happen to it is a loss of position, but if, despite all this, users stop visiting it, then there is already a reason to think about its temporary exclusion. But as soon as visitors visit the page, it appears again in the search, as evidenced by the first screenshot.

How to solve the problem of low-quality pages? To be honest, my vision is something like this - query analysis, removal of unnecessary keys, text optimization, backlinks to current pages.

There is an online store with a huge assortment (more than 4 million items). When I registered it in Yandex.Webmaster, I discovered that some of the pages (quite significant) were removed from Yandex search because they were not of sufficient quality. These are pages of different products that differ only in one number.
When I checked the TOP, I saw that exactly the same pages on similar sites were quietly in the index. That is, there are also sites that are full of similar products. But they are also no different. Only one number in Title and Description.

What can be done? To forget because all the pages cannot be in the index anyway, or to somehow try to improve the pages from the point of view of Yandex? I don’t know how, because it’s the same in the TOP. Minimal description and even the same photographs.

Hello. Give me a clear example. I think the whole problem lies in this: “These are pages of different products that differ only in one number.”

Hello! Yandex removes from the search pages with the following Titles: 1. Buy Porsche 955.044.80018 Decorative package:: Porsche 2. Buy Porsche 955.044.80053 Decorative package:: Porsche Cayenne Cayenne Both pages differ only in the headings H1 and Ttile. And then only for one digit. Yandex marked both of these pages as “Insufficient quality”. Although there are exactly the same pages in the TOP. There are also minimal characteristics and the picture is the same. How to solve the situation? Add text?

I think I understand you. I may disappoint you.

Such sites import automatically and create pages that fall under a small filter. On this moment The Yandex algorithm is not that good, but it improves every year. Such sites have no future. The problem here is automation and easy import. Search Algorithm can’t decide which of these “pages” to keep and which to remove. Since each one has almost the same thing. If we consider the history of Google, then Yandex is coming along the same path. In the end, your Website with these millions of pages will someday fall under a strong filter and one thing will happen: complete indexing of your website.

What can you do? You need to create unique pages, which differ in quality from your competitors, and not in the number of pages! You need to change every page (Titel, Meta description and another permalink) or hope for the best. Explore SEO basics.
Unfortunately, there is nothing to recommend here regarding SEO, since the import concept is built on quick money and spam. It's not significant. Sorry.

This is an online store. There is a large assortment, so there are many pages. What does it have to do with quality rather than quantity? If competitors have fewer products, they have fewer pages. What does it have to do with quick money and spam? The pages are similar, but different, so the products are almost the same. How does this even relate to spam? The site has the same structure as thousands of similar pages. There is everything the user needs: characteristics, price, short description. How then do you think the rest of us who know the basics of SEO make money? Do you fill out each card manually? Do not make me laugh. Thank you for demonstrating your “deep knowledge” here, but not helping me in any way.

And what is your quality if you have a million pages that look like duplicates of other pages for search engine. That's what I wanted to say! So this is the problem. Where do you get Taytel, Meta description, product description and permalink structure for products? Do you mean to say that you inserted all millions of products manually? I wanted to explain to you that importing and quantity of goods will not lead to anything good. What others do is not important...
“Do they fill out each card by hand?”
Yes, or they let people fill it out and, based on customer requests, they decide which products are important and which are not.
I wrote at the top that you won't like it. (Meaned from above *de-indexing)

The new Yandex Webmaster allows you to find problems on the site and correct shortcomings that interfere with ranking - this is a medical fact. To do this, Yasha offered a set of interesting tools for the holiday table, many of which were not available on the old service. I offer an overview of the Webmaster's capabilities to stimulate a good mood.

Tool overview

The new Webmaster's tools are divided into blocks, each of them opens into tabs. Here are the main menu items:

  1. - Diagnostics,
  2. - Search queries,
  3. - Indexing,
  4. - Setting up indexing,
  5. - Tools.

The last item “Access rights” is not an accordion, this item is not a nesting doll, it does not open. As intuition suggests, each block contains tools for solving highly specialized problems, for example, in the “Search Queries” block you can find out everything about the requests of users who went to the site, impressions.

Diagnostics

The first menu item is “Diagnostics”, it contains two items:

  1. - Site diagnostics,
  2. - Security and violations.

In the first point, they learn about problems on the site, for example, inaccessible pages. The page is being updated creakingly about how diesel works in the Arctic, so you shouldn’t hope that it will promptly display all the problems, but sometimes it comes to the rescue.

The Security and Violations page displays site security issues and some violations. Here you can see information about viruses, sanctions and other problems. Also, not everything is displayed - for example, implicit text spam is not shown, but AGS is displayed.

Search queries

There are two points here:

  1. - Statistics,
  2. - Latest requests.

“Statistics” indicates site requests for certain period, as well as requests from the top 3-10-50. The webmaster can see impressions, clicks and CTR requests. If the CTR is low, then you should think about changing the snippet or favicon, because something in the search results discourages the user from active actions regarding the site. For reference, the CTR of the top 3 is always higher, since users have not yet gotten tired of looking for answers in the depths of the search results. If the CTR is less than 2%, then it’s a disaster for the top 10 normal indicator from 3%.

Requests can be sorted by groups, metrics, and region.

The "Latest Requests" page is the same, but in relation to recent actions user in the search results. For the webmaster latest requests for impressions, clicks and CTR + added average position by request.

Sorting is also available here, you can add a filter or group queries.

Indexing

A more massive “Indexing” item, it contains the following tools:

  1. - Statistics,
  2. - Site structure,
  3. - URL checking,
  4. - Internal and external links.

The statistics page shows downloaded and excluded web documents, as well as pages in the search. The display of HTTP code errors and pages prohibited from indexing by the robot can be configured separately.

At the bottom there are statistics of the last crawls by the bot, you can see the document url, HTTP status and the date of the robot’s visit.

A useful item is “Pages in search”, because there you can see not only banality, but also find valuable information. The tool shows the number of added and excluded pages, sorting is available by latest changes and all pages. What's good for a webmaster's weakened nervous system? The tool shows the status of excluded pages, not the HTTP status, but the reason for the exclusion.

The two main reasons for exclusion from search are duplicates and “insufficient quality of the page.” If the number of excluded pages (except for those specifically deleted through 404) is large, then expect a deterioration in the site’s ranking. Use this tool, look at the reason for deleting pages and eliminate it for your own good. This is not difficult to do - look at the reason, go to the page and give it a humanoid appearance, and then send the document for re-indexing. Tested - it works and helps you get out of filters and not fall under sanctions.

The Site Structure tool is simple but useful. It shows the structure of the site and indicates the number of downloaded and indexed pages in each section and category. Strive to bring the second value closer to the first digit, for this purpose delete and close duplicates and garbage in robots.

The URL Checker tool allows you to check any page on your site for indexing. Nothing original. Added the URL, checked it, clicked “done” and saw whether it was in the index or not + HTTP status

The “Internal and External Links” tools are more interesting, although it is not clear why they were put in the “Indexation” section.

With the help of the “Internal Links” tool it is easy to see broken backends on internal pages indicating where the link is located and what page it leads to. Clean everything to zero and there will be happiness - no third party services are not needed - the Webmaster will help you find all the broken links.

  1. - Current and deleted links,
  2. - Grouping by sites,
  3. - Non-working backers.

Above indicated total number external links to the site and domains from which they link to you. Below is detailed information:

  1. - Donor's address,
  2. - TCI of the referring site,
  3. - Acceptor page,
  4. - Date of information update.

Information is updated after reference updates.

Setting up indexing

Here is a place for tools:

  1. - Sitemap files,
  2. - Site relocation,
  3. - Regionality,
  4. - Register of site names,
  5. - Quick links,
  6. - Original texts,
  7. - Products and prices.

In the Sitemap tool, you can see the sitemaps used, and immediately add a Sitemap in xml format.

In the “Site Moving” tab, they add www and HTTPS for the domain, and in the “Regionality” tool, the site regions for YAK, Yandex Directory and Webmaster are visible. In the last paragraph, you can specify the region, but it must be confirmed with a page that contains information about the region.

The next three points are clear to a schoolchild, and the “Products and Prices” tool will be useful for an online store. To do this, agree to the terms and connect the YML files. I don’t recommend climbing here without a programmer - it’s fraught with nerves and enemas.

Tools

Here, as the name suggests, are presented technical tools Webmasters Yandex. For the website owner or optimizer:

  1. - Sitemap analysis and Robots,
  2. - Server responses,
  3. - Removing URL,
  4. - Examination mobile adaptability pages,
  5. - Re-bypass,
  6. - Micro markup,
  7. - XML ​​validator.

Everything here is intuitively clear, Special attention I advise you to pay attention to the “Check” tool mobile pages” because it works a little differently than Google's responsiveness check. If you are promoting under Yandex and have large percentage mobile audience, then there should be no problems with displaying pages on tablets and smartphones.

The last item “Access Rights” allows you to delegate access rights to someone to view statistics through your Webmaster panel. Specify the Yandex login of your optimizer and delegate rights to it.