Read a detailed post about the update → Netpeak Spider 3.1: New Reports and Usability Improvements.
Hello everyone! Today I want to tell you about all the changes in Netpeak Spider 3.1! We've added several super useful reports and, of course, improved the interface of the tool. I can't wait to show you all the updates, so let's open the new Netpeak Spider and get started ;)
1. Site Structure Report
If you want to make an in-depth analysis of your or competitor's website in the context of the segments in the URL structure, our new 'Site structure' report is the thing you should try for this task.
Exported file will consist of all parameters that were selected in a sidebar in the 'Parameters' tab, but the most important of part of the report is the 'Section' column! It's similar to the numbered lists in any document and makes it easy to sort the structure and also quickly determine the number and uniqueness of segments in URL. We've also added this column to the extended copy feature, which is available in the 'Site Structure' tab in a sidebar.
If you see a cell in the 'URL' column marked in yellow, it means that this section does not have the main page. For example, you have a page example.com/tags/cats, but don't have a page example.com/tags, or it was blocked by considered indexation instructions → in this case, it will be marked yellow. I understand that this is not a critical issue, but it's better to be informed about such things in URL structure and make appropriate actions.
Note, that now in 'Site structure' and 'Overview' reports all hosts are sorted in 2 steps. First one is by dots (fewer dots mean higher in the list) and then in alphabetic order. All the subsequent segments in the URL are always sorted in alphabetical order.
But the most interesting thing about this feature is an opportunity to visualize the data. I will show you how to do it with XMind program. Select a necessary part of the structure in the exported report then copy it and paste without formatting to the program (Shift+Ctrl+V).
2. Redirects: Incoming Links and Final URLs
Ever since I've started working at Netpeak Software, the most frequent user request has always been a report about redirects which consists of:
- page which contains a link leading to a redirect
- where this link initially leaded (its anchor and status code)
- final redirect URL
I'm glad to say that this report is available now with additional useful information. Thus, you can send it as a task for developers or content managers to get rid of unnecessary redirects on your website. Redirect itself is not a critical problem but it's better to lead users and search robots right to the target page without time losses, as it's crucial for saving your crawling budget.
3. Scraping Summary in Single File
One of the most popular Netpeak Spider use cases is scraping different kinds of data from websites. People mostly scrape contact details, prices, goods characteristics, reviews, etc. Ways of using scraping are limited only by your imagination. That's why we needed to improve exporting this data to make further use of reports easier for the customers. That's how 'Scraping summary in a single file (XL)' and 'Data for scraping conditions in separate files (XL)' reports were born.
Here is how 'Scraping summary in single file (XL)' report looks like when you scrape data from Amazon product pages.
As you can see, each URL corresponds with only one row of the table and consists of all the data scraped from it. There is one useful advantage: when exporting this report in .XLSX format you'll see notes located in column headers. If you use custom scraper names (ex. 'Price'), you will see scraping conditions used for retrieving this data when hovering over the column headers.
4. Exporting Pending URLs
If you've got a task to get a complete list of URLs of your or competitors' websites quickly, Netpeak Spider 3.1 can solve it in just a few clicks. 'Pending URLs' report will have a list of all website pages and their depth from the initial URL even before the crawling is complete. By the way, before starting crawling, you can set up settings like scanning external links or subdomains that will also be considered in this report.
5. New Report Export Menu
Let's go deeper into each item of the new report export menu:
- Current table results → performs the same function as 'Export' button on the top of the main program tables. As a result, you get a report with data from current results table or a dashboard.
- Main reports set → exports the folder which includes 'All issues', 'All results', 'Site structure', 'Scraping summary in a single file (XL)', and 'All unique URLs & anchors (XL)' reports.
- All issues → it's a bulk export of all reports about issues that have been found during the crawling. If you don't want to export all of them, choose the necessary ones from the 'Issue reports' group.
- I've already told you about 'Site structure' and 'Pending URLs' reports, but if you overlooked them just re-watch this video.
- 'XL (extra large) reports from database' → documents from this export potentially can be very heavy because they contain scraping results (summary and each scraper in separate files) and reports with links (internal, external, and also unique URLs and anchors).
- 'All available reports (main + XL)' → it's the heaviest type of bulk export because it includes all available reports except 'Current table results'.
I also want to underline two awesome reports that are usually overlooked, as they are placed in 'XL (extra large) reports from database' group. First one is 'Unique URLs', where you can find all unique links and their number on a website (internal and external in separate files), but also the number of unique anchors that have been used for these links. It is useful when you want to find all external sites you refer to or how many different anchors are used for some internal page of the analyzed site.
The second one is 'Unique anchors' and it can be used to see the anchor texts and their number. You will find the data grouped by anchors, and we've also added the number of links with such anchor texts as a separate column. It's useful for more in-depth analysis of the site internal linking and anchor list used for links to certain groups of pages.
6. Expected Crawling Duration
If you are not sure if you have enough time to grab a burrito from the nearest Chipotle during site crawling, Netpeak Spider will help you to make this decision. In the latest update, we've implemented a crawling duration forecast with an easy algorithm: we take a number of pending URLs and divide it by crawling speed. Looks like a question from SATs: Netpeak Spider train leaves an initial URL at the crawling speed. At what time will the SEO specialist see the complete report if it's ten thousand pages left? The expected time lift is located in the status bar.
7. Custom Table Configuration
We like to adjust everything around us for our comfort, and now our tables are a part of your working environment that can be customized too!
Let me show what you can configure:
- Column width.
- Relative position of columns.
- Data sorting in columns.
- Data grouping by the value of any parameter or a group of parameters. You can drag the column heading to the special zone above the table.
- Freezed columns.
And all these customizations can be applied to each table separately! Each issue report can be displayed the way you want it to be displayed. The only thing I want to note is 'Overview', 'Site structure', 'Scraping' tables can't be customized separately because Netpeak Spider opens them in filtered results tab. Thus, you can use only one simultaneous appearance for them.
8. Synchronizing Tables with Parameters
Users of Netpeak Checker 3.1 already know this feature but now it's also available in Netpeak Spider. If you don't want to see the whole data, but surf only among necessary parameters, follow these 3 steps:
- When the crawling is complete, save the project.
- Untick unnecessary parameters in a sidebar.
- Click on 'Sync the table with selected parameters' button → here we go, only selected parameters will be displayed in the report. By the way, as you can see we replaced function name with icons here. Now you will see the explanation of them in a tooltip when hovering over the button.
If you forgot what is hidden, you can open a previously saved project. Also, you can save currently analyzed parameters as a template. After some manipulations with syncing, you can use it and click on 'Sync the table with selected parameters' to show all data again (or for the next analysis). For the best experience use all functionality of the program ;)
9. 'Filter by Value' Option
The same as the previous feature, it was implemented in Netpeak Checker 3.1 and then copied to Netpeak Spider. To make a long story short, it helps you filter data by a value of one or several cells in one row. For example, you can quickly filter all the pages that are placed deeper than 4 clicks from the initial URL.
Brief Summary
In Netpeak Spider 3.1:
- Most of the changes are about reports: we added 4 new options and gathered all available reports in handy groups.
- You can customize all program tables according to your preferences and it will be saved for further use after the program is closed.
- Crawling duration forecast displayed in status bar.
- We copied 'Synchronizing tables with parameters' ('Parameters' tab in a sidebar) and 'Filter by value' (context menu in the majority of tables) option from Netpeak Checker 3.1 to the latest update of Netpeak Spider.
- More than 20 interface and usability changes to make your program experience better.
Thanks for your attention and time, guys! I would be happy if you write in the comments about a feature in our program which you have never known about before this video. Alright, it's my time to go back to work, wish you a good PageSpeed Insights score and pretty snippets that sell ;)
Digging This Update? Let's Discuss Netpeak Spider Perks in Person
Was this article helpful?
That’s Great!
Thank you for your feedback
Sorry! We couldn't be helpful
Thank you for your feedback
Feedback sent
We appreciate your effort and will try to fix the article