Skip to content

Releases: t1gor/Robots.txt-Parser-Class

Version 0.2.5

13 Sep 12:41
ac41590
Compare
Choose a tag to compare

Bug fixes

  • Fix for rules containing [ and ]. #85

Version 0.2.4

21 Jul 21:21
Compare
Choose a tag to compare

Bug fixes

  • $parser->setUserAgent($string) no longer being ignored
  • Fix for rules containing an end anchor $

Other improvements

  • Improved User-agent parser and group determiner

Version 0.2.3

10 Aug 21:36
Compare
Choose a tag to compare
  • PHP 7.1 compatibility
  • Rendering function - To render/prettify robots.txt files
  • Reformatted and reordered code

Version 0.2.2

27 Apr 13:55
Compare
Choose a tag to compare
  • An warning is generated if the User-agent format isn't supported, eg when using the full browser-like user-agent string instead of just name/version. #64
  • setHttpStatusCode now returns bool, true on success or false for any invalid http status code.

Version 0.2.1

24 Feb 18:53
Compare
Choose a tag to compare
  • Multiple user-agents in a single group, now share the same rules. #60

Version 0.2

31 Jan 19:52
Compare
Choose a tag to compare

Now also supports:

  • Yandex robots.txt specifications.
  • W3C Recommendation robots.txt specifications.
  • The Web Robots Pages robots.txt specifications.
  • The Clean-Param directive.
  • Inline directives.

Features and enhancements:

  • Clean-Param array export ability.
  • Full URLs in isAllowed and isDisallowed now supported.

Ref. Wiki for more information.

Version 0.1

26 Dec 05:21
Compare
Choose a tag to compare

First stable feature complete release!

  • Full support for Google's Robots.txt Specifications

Almost every aspect of the official robots.txt specifications has been tested and lots of bugs has been fixed.

Version 0.1 beta

25 Dec 23:04
Compare
Choose a tag to compare
Version 0.1 beta Pre-release
Pre-release

New functions

  • Handling HTTP result codes
  • Order of precedence for user-agents

Crawl delay and cache delay support

14 Dec 15:51
Compare
Choose a tag to compare
Pre-release

Thanks to our contributors, we can now support the following directives:

  1. Crawl Delay
  2. Cache Delay
  3. Clean Param

Code coverage should also be improved, at least a bit 👍

Great job, guys!

Special thanks to @JanPetterMG !

Current code base released as DEV

15 Jun 07:00
Compare
Choose a tag to compare
v0.0.1-dev

codeclimate