Tags: qiubits2007/XML-Sitemap
Tags
Add files via upload ### Added - Structured logging with timestamp, log level and message per entry - New `flattenLog()` groups output by category (INDEXED, SKIPPED, BLOCKED, ERRORS, etc.) - Export as `crawl_log.json`, `crawl_log.txt` and `crawl_log.html` - New `crawl_graph.json` for visual map data - New interactive `crawl_map.html` with filters for depth, status, type - Configurable log email sender via `--from` - Full crawl report as `.txt` attachment in mail ### Changed - `pingSearchEngines()` adjusted: Google & Bing ping endpoints deprecated - Ping to search engines now optional and safely logged - `addLog()` consistently supports levels: `info`, `error`, `skipped`, `indexed`, `blocked`, `sitemap`, `ping`, etc. ### Fixed - Duplicate log output on resume - Issues with incorrect sitemap ping URLs - General structure cleanup and docblocks added
Add files via upload - Introduced `--splitbysite` CLI option to generate separate sitemap files per domain - Added automatic creation of `sitemap_index.xml` referencing all domain sitemaps - Updated search engine ping logic to use sitemap index if split mode is active - Improved logging for ping responses and sitemap index generation - Updated README.md and CHANGELOG.md with new documentation and options