diff options
| author | Edi Weitz <edi@agharta.de> | 2008-07-23 11:29:40 +0000 |
|---|---|---|
| committer | Edi Weitz <edi@agharta.de> | 2008-07-23 11:29:40 +0000 |
| commit | 2974af4010d8ce8779324b2bdecf0940b2c352c8 (patch) | |
| tree | a0b2f93bec81064dbb78f54b489faed74609ae03 /doc | |
| parent | bf6913769fbdaf7c04061ee583036b1e2492fa42 (diff) | |
Import 1.4.1 version of CL-PPCRE
git-svn-id: svn://bknr.net/svn/trunk/thirdparty/cl-ppcre@3577 4281704c-cde7-0310-8518-8e2dc76b1ff0
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/index.html | 412 |
1 files changed, 305 insertions, 107 deletions
diff --git a/doc/index.html b/doc/index.html index 9f05d1e..eefdc09 100644 --- a/doc/index.html +++ b/doc/index.html @@ -3,21 +3,30 @@ <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> - <title>CL-PPCRE - portable Perl-compatible regular expressions for Common Lisp</title> + <title>CL-PPCRE - Portable Perl-compatible regular expressions for Common Lisp</title> <style type="text/css"> pre { padding:5px; background-color:#e0e0e0 } - a { text-decoration: none; } - a.none:hover { border:1px solid white; } - a { border:1px solid white; } - a:hover { border: 1px solid black; } - a.noborder { border:0px } - a.noborder:hover { border:0px } + h3, h4 { text-decoration: underline; } + a { text-decoration: none; padding: 1px 2px 1px 2px; } + a:visited { text-decoration: none; padding: 1px 2px 1px 2px; } + a:hover { text-decoration: none; padding: 1px 1px 1px 1px; border: 1px solid #000000; } + a:focus { text-decoration: none; padding: 1px 2px 1px 2px; border: none; } + a.none { text-decoration: none; padding: 0; } + a.none:visited { text-decoration: none; padding: 0; } + a.none:hover { text-decoration: none; border: none; padding: 0; } + a.none:focus { text-decoration: none; border: none; padding: 0; } + a.noborder { text-decoration: none; padding: 0; } + a.noborder:visited { text-decoration: none; padding: 0; } + a.noborder:hover { text-decoration: none; border: none; padding: 0; } + a.noborder:focus { text-decoration: none; border: none; padding: 0; } + pre.none { padding:5px; background-color:#ffffff } </style> + <meta name="description" content="Fast and portable perl-compatible regular expressions for Common Lisp."> </head> <body bgcolor=white> -<h2>CL-PPCRE - portable Perl-compatible regular expressions for Common Lisp</h2> +<h2>CL-PPCRE - Portable Perl-compatible regular expressions for Common Lisp</h2> <blockquote> <br> <br><h3>Abstract</h3> @@ -29,24 +38,20 @@ which has the following features: <li>It is <b>compatible with Perl</b>. (Well - as far as you can be compatible with a language defined by its ever-changing -implementation. Currently, as of December 2002, CL-PPCRE is more -compatible with the regex semantics of Perl 5.8.0 than, say, -Perl 5.6.1 is...:) It even correctly parses and applies <a -href="http://www.oreilly.com/catalog/regex2/">Jeffrey Friedl's</a> -famous 6600-byte long RFC822 address pattern. - -<li>It is <b>fast</b>. If compiled with <a -href="http://www.cons.org/cmucl/">CMUCL</a> it <a -href="#performance">outperforms</a> Perl's highly optimized regex engine (written -in C) which to my knowledge is faster than most other regex engines -around. If compiled with <a -href="http://clisp.sourceforge.net/">CLISP</a> it is still comparable -to CLISP's own regex implementation which is also written in -C. +implementation. As of December 2002 CL-PPCRE was <a href="#perl">more +compatible</a> with the regex semantics of Perl 5.8.0 than, say, +Perl 5.6.1 was...) It even correctly parses and +applies <a href="http://www.oreilly.com/catalog/regex2/">Jeffrey +Friedl's</a> famous 6600-byte long RFC822 address pattern. + +<li>It is <b>fast</b>. Used with a Lisp compiler which compiles to +native code it is <a href="#performance">on par</a> with Perl's highly +optimized regex engine (written in C) which to my knowledge is faster +than most other regex engines around. <li>It is <b>portable</b>, i.e. the code aims to be strictly <a href="http://www.lispworks.com/documentation/HyperSpec/Front/index.htm">ANSI-compliant</a>. If -you encounter any deviations this is an error and should be +you encounter any deviations, this is an error and should be reported to <a href="#mail">the mailing list</a>. CL-PPCRE has been successfully tested with the following Common Lisp implementations: @@ -68,7 +73,7 @@ successfully tested with the following Common Lisp implementations: </ul> -If you succeed in using CL-PPCRE on other platforms please <a +If you succeed in using CL-PPCRE on other platforms, please <a href="#mail">let us know</a>. <br> @@ -117,7 +122,9 @@ license</b></a> so you can basically do with it whatever you want. </ul> CL-PPCRE has been used successfully in various applications like <a -href="http://nostoc.stanford.edu/Docs/">BioLingua</a>, <a +href="http://nostoc.stanford.edu/Docs/">BioBike</a>, +<a href="http://clutu.com/">clutu</a>, +<a href="http://www.hpc.unm.edu/~download/LoGS/">LoGS</a>, <a href="http://cafespot.net/">CafeSpot</a>, <a href="http://www.eboy.com/">Eboy</a>, or <a href="http://weitz.de/regex-coach/">The Regex Coach</a>. @@ -150,9 +157,10 @@ href="http://weitz.de/regex-coach/">The Regex Coach</a>. <li><a href="#regex-replace-all"><code>regex-replace-all</code></a> <li><a href="#regex-apropos"><code>regex-apropos</code></a> <li><a href="#regex-apropos-list"><code>regex-apropos-list</code></a> - <li><a href="#regex-char-code-limit"><code>*regex-char-code-limit*</code></a> - <li><a href="#use-bmh-matchers"><code>*use-bmh-matchers*</code></a> + <li><a href="#*regex-char-code-limit*"><code>*regex-char-code-limit*</code></a> + <li><a href="#*use-bmh-matchers*"><code>*use-bmh-matchers*</code></a> <li><a href="#*allow-quoting*"><code>*allow-quoting*</code></a> + <li><a href="#*allow-named-registers*"><code>*allow-named-registers*</code></a> <li><a href="#quote-meta-chars"><code>quote-meta-chars</code></a> <li><a href="#ppcre-error"><code>ppcre-error</code></a> <li><a href="#ppcre-invocation-error"><code>ppcre-invocation-error</code></a> @@ -192,15 +200,14 @@ href="http://weitz.de/regex-coach/">The Regex Coach</a>. CL-PPCRE together with this documentation can be downloaded from <a href="http://weitz.de/files/cl-ppcre.tar.gz">http://weitz.de/files/cl-ppcre.tar.gz</a>. The -current version is 1.2.12. A <a -href="CHANGELOG">CHANGELOG</a> is available. +current version is 1.4.1. <p> -If you're on <a href="http://www.debian.org/">Debian</a> you should +If you're on <a href="http://www.debian.org/">Debian</a>, you should probably use the <a -href="http://packages.debian.org/cgi-bin/search_packages.pl?keywords=cl-ppcre&searchon=names&version=all&release=all">cl-ppcre +href="http://packages.debian.org/cgi-bin/search_packages.pl?keywords=cl-ppcre&searchon=names&version=all&release=all">cl-ppcre Debian package</a> which is available thanks to <a href="http://pvaneynd.mailworks.org/">Peter van Eynde</a> and <a href="http://b9.com/">Kevin Rosenberg</a>. There's also a port -for <a href="http://www.cliki.net/gentoo">Gentoo Linux</a> thanks to Matthew Kennedy and a <a href="http://www.freebsd.org/cgi/url.cgi?ports/textproc/cl-ppcre/pkg-descr">FreeBSD port</a> thanks to Henrik Motakef. +for <a href="http://www.gentoo.org/proj/en/common-lisp/index.xml">Gentoo Linux</a> thanks to Matthew Kennedy and a <a href="http://www.freebsd.org/cgi/url.cgi?ports/textproc/cl-ppcre/pkg-descr">FreeBSD port</a> thanks to Henrik Motakef. Installation via <a href="http://www.cliki.net/asdf-install">asdf-install</a> should as well be possible. @@ -214,7 +221,7 @@ directory start your Lisp image and evaluate the form equivalent one for asdf) which should compile and load the whole system. <p> -If for some reason you don't want to use MK:DEFSYSTEM or asdf you +If for some reason you don't want to use MK:DEFSYSTEM or asdf, you can just <code>LOAD</code> the file <code>load.lisp</code> or you can also get away with something like this: @@ -239,35 +246,46 @@ cat {packages,specials,util,errors,lexer,parser,regex-class,convert,optimize,clo your platform.) <p> Note that there is <em>no</em> public CVS repository for CL-PPCRE - the repository at <a href="http://common-lisp.net/">common-lisp.net</a> is out of date and not in sync with the (current) version distributed from <a href="http://weitz.de/">weitz.de</a>. +<p> +Luís Oliveira maintains a <a href="http://darcs.net/">darcs</a> +repository of CL-PPCRE +at <a +href="http://common-lisp.net/~loliveira/ediware/">http://common-lisp.net/~loliveira/ediware/</a>. <br> <br><h3><a name="mail" class=none>Support and mailing lists</a></h3> For questions, bug reports, feature requests, improvements, or patches -please use the <a -href="http://common-lisp.net/mailman/listinfo/cl-ppcre-devel">cl-ppcre-devel -mailing list</a>. If you want to be notified about future releases -subscribe to the <a -href="http://common-lisp.net/mailman/listinfo/cl-ppcre-announce">cl-ppcre-announce +please use +the <a href="http://common-lisp.net/mailman/listinfo/cl-ppcre-devel">cl-ppcre-devel +mailing list</a>. If you want to be notified about future releases, +subscribe to +the <a href="http://common-lisp.net/mailman/listinfo/cl-ppcre-announce">cl-ppcre-announce mailing list</a>. These mailing lists were made available thanks to the services of <a href="http://common-lisp.net/">common-lisp.net</a>. +Terrence Brannon has created a Google group for the list which is +at <a href="http://groups.google.com/group/cl-ppcre">http://groups.google.com/group/cl-ppcre</a>. +<p> +If you want to send patches, please <a href="http://weitz.de/patches.html">read this first</a>. <br> <br><h3><a class=none name="dict">The CL-PPCRE dictionary</a></h3> CL-PPCRE exports the following symbols: <p><br>[Method] -<br><a class=none name="create-scanner"><b>create-scanner</b> <i>(string string)<tt>&key</tt> case-insensitive-mode multi-line-mode single-line-mode extended-mode destructive</i> => <i>scanner</i></a> +<br><a class=none name="create-scanner"><b>create-scanner</b> <i>(string string)<tt>&key</tt> case-insensitive-mode multi-line-mode single-line-mode extended-mode destructive</i> => <i>scanner, register-names</i></a> <blockquote><br> Accepts a string which is a regular expression in Perl syntax and returns a closure which will scan strings for this -regular expression. The mode keyword arguments are equivalent to the +regular expression. The second value is only returned if <a href="#*allow-named-registers*"><code>*ALLOW-NAMED-REGISTERS*</code></a> is <i>true</i>. It represents a list of strings mapping registers to their respective names - the first element stands for first register, the second element for second register, etc. You have to store this value if you want to map a register number to its name later as <i>scanner</i> doesn't capture any information about register names. If a register isn't named, it has NIL as its name. +<p> +The mode keyword arguments are equivalent to the <code>"imsx"</code> modifiers in Perl. The <code>destructive</code> keyword will be ignored. <p> The function accepts most of the regex syntax of Perl 5 as described in <a -href="http://www.perldoc.com/perl5.8.0/pod/perlre.html"><code>man +href="http://perldoc.perl.org/perlre.html"><code>man perlre</code></a> including extended features like non-greedy repetitions, positive and negative look-ahead and look-behind assertions, "standalone" subexpressions, and conditional @@ -308,6 +326,8 @@ codes), <code>\c[</code> (control characters), <code>\w</code>, Since version 0.6.0 CL-PPCRE also supports Perl's <code>\Q</code> and <code>\E</code> - see <a href="#*allow-quoting*"><code>*ALLOW-QUOTING*</code></a> below. Make sure you also read <a href="#quote">the relevant section</a> in "<a href="#bugs">Bugs and problems</a>." <p> +Since version 1.3.0 CL-PPCRE offers support for <a href="http://www.franz.com/support/documentation/7.0/doc/regexp.htm#regexp-new-capturing-2">AllegroCL's</a> <code>(?<name>"<regex>")</code> named registers and <code>\k<name></code> back-references syntax, have a look at <a href="#*allow-named-registers*"><code>*ALLOW-NAMED-REGISTERS*</code></a> for details. +<p> The keyword arguments are just for your convenience. You can always use embedded modifiers like <code>"(?i-s)"</code> instead.</blockquote> @@ -319,7 +339,7 @@ In this case <code><i>function</i></code> should be a scanner returned by anothe </blockquote> <p><br>[Method] -<br><a class=none name="create-scanner2"><b>create-scanner</b> <i>(parse-tree t)<tt>&key</tt> case-insensitive-mode multi-line-mode single-line-mode extended-mode destructive</i> => <i>scanner</i></a> +<br><a class=none name="create-scanner2"><b>create-scanner</b> <i>(parse-tree t)<tt>&key</tt> case-insensitive-mode multi-line-mode single-line-mode extended-mode destructive</i> => <i>scanner, register-names</i></a> <blockquote><br> This is similar to <a href="#create-scanner"><code>CREATE-SCANNER</code></a> for regex strings above but @@ -405,7 +425,7 @@ stands for a register or a parse tree which is a look-ahead or look-behind assertion. See the entry for <code>(?(<<i>condition</i>>)<<i>yes-pattern</i>>|<<i>no-pattern</i>>)</code> in <a -href="http://www.perldoc.com/perl5.8.0/pod/perlre.html#Extended-Patterns"><code>man +href="http://perldoc.perl.org/perlre.html#Extended-Patterns"><code>man perlre</code></a> for the semantics of this construct. If <code><<i>parse-tree</i>></code> is an alternation is <em>must</em> enclose exactly one or two parse trees where the second @@ -439,9 +459,19 @@ to the Perl regex string <code>"(?:ab)??"</code>. register group. As usual, registers are counted from left to right beginning with 1. -<li><code>(:BACK-REFERENCE <<i>number</i>>)</code> where -<code><<i>number</i>></code> is a positive integer is a back-reference to a -register group. +<li><code>(:NAMED-REGISTER <<i>name</i>> <<i>parse-tree</i>>)</code> is a named capturing +register group. Acts as <code>:REGISTER</code>, but assigns <code><<i>name</i>></code> to a register too. This <code><<i>name</i>></code> can be later referred to via <code>:BACK-REFERENCE</code>. Names are case-sensitive and don't need to be unique. See <a href="#*allow-named-registers*"><code>*ALLOW-NAMED-REGISTERS*</code></a> for details. + + +<li><code>(:BACK-REFERENCE <<i>ref</i>>)</code> is a +back-reference to a register group. <code><<i>ref</i>></code> is +a positive integer or a string denoting a register name. If there are +several registers with the same name, the regex engine tries to +successfully match at least of them, starting with the most recently +seen register continuing to the least recently seen one, until a match +is found. See <a +href="#*allow-named-registers*"><code>*ALLOW-NAMED-REGISTERS*</code></a> +for more information. <li><a class=none name="filterdef"><code>(:FILTER <<i>function</i>> <tt>&optional</tt> <<i>length</i>>)</code></a> where @@ -451,6 +481,11 @@ designator</a> and <code><<i>length</i>></code> is a non-negative integer or <code>NIL</code> is a user-defined <a href="#filters">filter</a>. +<li><code>(:REGEX <<i>string</i>>)</code> where +<code><<i>string</i>></code> is an +embedded <a href="#create-scanner">regular expression in Perl +syntax</a>. + <li><code>(:CHAR-CLASS|:INVERTED-CHAR-CLASS {<<i>item</i>>}*)</code> where <code><<i>item</i>></code> is either a character, a <em>character range</em>, or a symbol for a @@ -483,11 +518,11 @@ doesn't make sense if <code>CREATE-SCANNER</code> is applied to parse trees and will signal an error. <p> If <code>destructive</code> is not <code>NIL</code> (the default is -<code>NIL</code>) the function is allowed to destructively modify +<code>NIL</code>), the function is allowed to destructively modify <code><i>parse-tree</i></code> while creating the scanner. <p> If you want to find out how parse trees are related to Perl regex -strings you should play around with +strings, you should play around with <code>CL-PPCRE::PARSE-STRING</code> - a function which converts Perl regex strings to parse trees. Here are some examples: @@ -514,7 +549,7 @@ regex strings to parse trees. Here are some examples: <p><br>[Accessor] <br><a class="none" name="parse-tree-synonym"><b>parse-tree-synonym</b> <i>symbol</i> => <i>parse-tree</i> -<br><tt>(setf (</tt><b>parse-tree-synonym</b> <i>symbol</i>) <i>new-parse-tree</i><tt>)</tt></a> +<br><tt>(setf (</tt><b>parse-tree-synonym</b> <i>symbol</i><tt>)</tt> <i>new-parse-tree</i><tt>)</tt></a> </p><blockquote><br> Any symbol (unless it's a keyword with a special meaning in parse @@ -568,7 +603,7 @@ so you can write code like this: <pre> (define-parse-tree-synonym a-z - (:char-class (:range #\a #\z) (:range #\a #\z))) + (:char-class (:range #\a #\z) (:range #\A #\Z))) (define-parse-tree-synonym a-z* (:greedy-repetition 0 nil a-z)) @@ -590,19 +625,21 @@ href="#scan"><code>SCAN</code></a><b>.</b> -<p><br>[Standard Generic Function] +<p><br>[Generic Function] <br><a class=none name="scan"><b>scan</b> <i>regex target-string <tt>&key</tt> start end</i> => <i>match-start, match-end, reg-starts, reg-ends</i></a> <blockquote><br> -Searches the string <code><i>target-string</i></code> from -<code><i>start</i></code> (which defaults to 0) to +Searches the string <code><i>target-string</i></code> +from <code><i>start</i></code> (which defaults to 0) to <code><i>end</i></code> (which default to the length of <code><i>target-string</i></code>) and tries to match <code><i>regex</i></code>. On success returns four values - the start of the match, the end of the match, and two arrays denoting the beginnings and ends of register matches. On failure returns -<code>NIL</code>. <code><i>target-string</i></code> will be coerced to a -simple string if it isn't one already. +<code>NIL</code>. <code><i>target-string</i></code> will be coerced +to a simple string if it isn't one already. (There's another keyword +parameter <code><i>real-start-pos</i></code>. This one should +<em>never</em> be set from user code - it is only used internally.) <p> <code>SCAN</code> acts as if the part of <code><i>target-string</i></code> between <code><i>start</i></code> @@ -691,7 +728,7 @@ and <code>FN</code> is a <a href="http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_f.htm#function_designator">function designator</a> (which is evaluated) denoting a function which is to be applied to the string before the result is bound to <code>VAR</code>. To make this even more convenient the form <code>(FN VAR1 ...VARn)</code> can be used as an abbreviation for -<code>(FN VAR1) ... (FN VARn). +<code>(FN VAR1) ... (FN VARn)</code>. <p> If there is no match, the <code><i>statement*</i></code> forms are <em>not</em> executed. For each element of @@ -739,7 +776,7 @@ returns <code><i>result-form</i></code> if provided or <code>NIL</code> otherwise. An implicit block named <code>NIL</code> surrounds <code>DO-SCANS</code>; <code>RETURN</code> may be used to terminate the loop immediately. If <code><i>regex</i></code> matches -an empty string the scan is continued one position behind this match. +an empty string, the scan is continued one position behind this match. <p> This is the most general macro to iterate over all matches in a target string. See the source code of <a @@ -830,7 +867,7 @@ groups. For each element of group. After the last match, returns <code><i>result-form</i></code> if provided or <code>NIL</code> otherwise. An implicit block named <code>NIL</code> surrounds <code>DO-REGISTER-GROUPS</code>; <code>RETURN</code> may be used to terminate the loop immediately. If <code><i>regex</i></code> matches -an empty string the scan is continued one position behind this +an empty string, the scan is continued one position behind this match. If <code><i>sharedp</i></code> is true, the substrings may share structure with <code><i>target-string</i></code>. <p>Example: @@ -916,7 +953,7 @@ simply be left out, otherwise they will show up as elements returned - registers aren't counted. If <code><i>limit</i></code> is <code>NIL</code> (or 0 which is equivalent), trailing empty strings are removed from the result list. -If <code><i>regex</i></code> matches an empty string the scan is +If <code><i>regex</i></code> matches an empty string, the scan is continued one position behind this match. If <code><i>sharedp</i></code> is true, the substrings may share structure with <code><i>target-string</i></code>. <p> @@ -970,12 +1007,14 @@ frob") <p><br>[Function] -<br><a class=none name="regex-replace"><b>regex-replace</b> <i>regex target-string replacement <tt>&key</tt> start end preserve-case simple-calls</i> => <i>list</i></a> +<br><a class=none name="regex-replace"><b>regex-replace</b> <i>regex target-string replacement <tt>&key</tt> start end preserve-case simple-calls element-type</i> => <i>string, matchp</i></a> <blockquote><br> Try to match <code><i>target-string</i></code> between <code><i>start</i></code> and <code><i>end</i></code> against <code><i>regex</i></code> and replace the first match with -<code><i>replacement</i></code>. +<code><i>replacement</i></code>. Two values are returned; the modified +string, and <code>T</code> if <code><i>regex</i></code> matched or +<code>NIL</code> otherwise. <p> <code><i>replacement</i></code> can be a string which may contain the special substrings <code>"\&"</code> for the whole @@ -1024,35 +1063,61 @@ will always be a <a href="http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_f.htm#fresh">fresh</a> string, even if <code><i>regex</i></code> doesn't match. <p> +<code><i>element-type</i></code> specifies +the <a +href="http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_a.htm#array_element_type">array +element type</a> of the string which is returned, the default +is <a +href="http://www.lispworks.com/documentation/lw50/LWRM/html/lwref-346.htm"><code>LW:SIMPLE-CHAR</code></a> +for LispWorks +and <a +href="http://www.lispworks.com/documentation/HyperSpec/Body/t_ch.htm"><code>CHARACTER</code></a> +for other Lisps. +<p> Examples: <pre> * (cl-ppcre:regex-replace "fo+" "foo bar" "frob") "frob bar" +T * (cl-ppcre:regex-replace "fo+" "FOO bar" "frob") "FOO bar" +NIL * (cl-ppcre:regex-replace "(?i)fo+" "FOO bar" "frob") "frob bar" +T * (cl-ppcre:regex-replace "(?i)fo+" "FOO bar" "frob" :preserve-case t) "FROB bar" +T * (cl-ppcre:regex-replace "(?i)fo+" "Foo bar" "frob" :preserve-case t) "Frob bar" +T * (cl-ppcre:regex-replace "bar" "foo bar baz" "[frob (was '\\&' between '\\`' and '\\'')]") "foo [frob (was 'bar' between 'foo ' and ' baz')] baz" +T * (cl-ppcre:regex-replace "bar" "foo bar baz" '("[frob (was '" :match "' between '" :before-match "' and '" :after-match "')]")) "foo [frob (was 'bar' between 'foo ' and ' baz')] baz" +T + +* (cl-ppcre:regex-replace "(be)(nev)(o)(lent)" + "benevolent: adj. generous, kind" + #'(lambda (match &rest registers) + (format nil "~A [~{~A~^.~}]" match registers)) + :simple-calls t) +"benevolent [be.nev.o.lent]: adj. generous, kind" +T </pre></blockquote> <p><br>[Function] -<br><a class=none name="regex-replace-all"><b>regex-replace-all</b> <i>regex target-string replacement <tt>&key</tt> start end preserve-case simple-calls</i> => <i>list</i></a> +<br><a class=none name="regex-replace-all"><b>regex-replace-all</b> <i>regex target-string replacement <tt>&key</tt> start end preserve-case simple-calls element-type</i> => <i>string, matchp</i></a> <blockquote><br> Like <a href="#regex-replace"><code>REGEX-REPLACE</code></a> but replaces all matches. @@ -1062,9 +1127,11 @@ Examples: <pre> * (cl-ppcre:regex-replace-all "(?i)fo+" "foo Fooo FOOOO bar" "frob" :preserve-case t) "frob Frob FROB bar" +T * (cl-ppcre:regex-replace-all "(?i)f(o+)" "foo Fooo FOOOO bar" "fr\\1b" :preserve-case t) "froob Frooob FROOOOB bar" +T * (let ((qp-regex (cl-ppcre:create-scanner "[\\x80-\\xff]"))) (defun encode-quoted-printable (string) @@ -1079,6 +1146,7 @@ ENCODE-QUOTED-PRINTABLE * (encode-quoted-printable "Fête Sørensen naïve Hühner Straße") "F=EAte S=F8rensen na=EFve H=FChner Stra=DFe" +T * (let ((url-regex (cl-ppcre:create-scanner "[^a-zA-Z0-9_\\-.]"))) (defun url-encode (string) @@ -1093,6 +1161,7 @@ URL-ENCODE * (url-encode "Fête Sørensen naïve Hühner Straße") "F%EAte%20S%F8rensen%20na%EFve%20H%FChner%20Stra%DFe" +T * (defun how-many (target-string start end match-start match-end reg-starts reg-ends) (declare (ignore start end match-start match-end)) @@ -1104,6 +1173,7 @@ HOW-MANY "foo{...}bar{.....}{..}baz{....}frob" (list "[" 'how-many " dots]")) "foo[3 dots]bar[5 dots][2 dots]baz[4 dots]frob" +T * (let ((qp-regex (cl-ppcre:create-scanner "[\\x80-\\xff]"))) (defun encode-quoted-printable (string) @@ -1120,6 +1190,7 @@ ENCODE-QUOTED-PRINTABLE * (encode-quoted-printable "Fête Sørensen naïve Hühner Straße") "F=EAte S=F8rensen na=EFve H=FChner Stra=DFe" +T * (defun how-many (match first-register) (declare (ignore match)) @@ -1132,6 +1203,7 @@ HOW-MANY :simple-calls t) "foo[3 dots]bar[5 dots][2 dots]baz[4 dots]frob" +T </pre></blockquote> <p><br>[Function] @@ -1217,18 +1289,18 @@ Example (continued from above): </pre></blockquote> <p><br>[Special variable] -<br><a class=none name="regex-char-code-limit"><b>*regex-char-code-limit*</b></a> +<br><a class=none name="*regex-char-code-limit*"><b>*regex-char-code-limit*</b></a> <blockquote><br>This variable controls whether scanners take into -account all characters of your CL implementation or only those the <a +account all characters of your CL implementation or only those +the <a href="http://www.lispworks.com/documentation/HyperSpec/Body/f_char_c.htm#char-code"><code>CHAR-CODE</code></a> -of which is not larger than its value. It is only relevant if the -regular expression contains certain character classes. The default is +of which is not larger than its value. The default is <a href="http://www.lispworks.com/documentation/HyperSpec/Body/v_char_c.htm"><code>CHAR-CODE-LIMIT</code></a>, and you might see significant speed and space improvements during scanner <em>creation</em> if, say, your target strings only contain <a -href="http://wwwwbs.cs.tu-berlin.de/user/czyborra/charsets/">ISO-8859-1</a> +href="http://czyborra.com/charsets/iso8859.html">ISO-8859-1</a> characters and you're using an implementation like AllegroCL, CLISP, LispWorks, or SBCL where <code>CHAR-CODE-LIMIT</code> has a value much higher than 256. The <a href="#test">test suite</a> will @@ -1259,8 +1331,8 @@ Allocation = 3336 bytes standard / 8338 bytes fixlen #<closure 206569DA> </pre> <p> -Note: Due to the nature of <code>LOAD-TIME-VALUE</code> and the <a -href="#compiler-macro">compiler macro for <code>SCAN</code></a> some +Note: Due to the nature of <a href="http://www.lispworks.com/documentation/HyperSpec/Body/s_ld_tim.htm"><code>LOAD-TIME-VALUE</code></a> and the <a +href="#compiler-macro">compiler macro for <code>SCAN</code> and other functions</a>, some scanners might be created in a <a href="http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_n.htm#null_lexical_environment">null lexical environment</a> at load time or at compile time so be careful @@ -1269,11 +1341,11 @@ time. The default value should always yield correct results unless you play dirty tricks with implementation-dependent behaviour, though.</blockquote> <p><br>[Special variable] -<br><a class=none name="use-bmh-matchers"><b>*use-bmh-matchers*</b></a> +<br><a class=none name="*use-bmh-matchers*"><b>*use-bmh-matchers*</b></a> <blockquote><br>Usually, the scanners created by <a href="#create-scanner"><code>CREATE-SCANNER</code></a> (or -implicitely by other functions and macros) will use fast <a +implicitly by other functions and macros) will use fast <a href="http://www-igm.univ-mlv.fr/~lecroq/string/node18.html">Boyer-Moore-Horspool matchers</a> to check for constant strings at the start or end of the regular expression. If <code>*USE-BMH-MATCHERS*</code> is @@ -1286,8 +1358,8 @@ href="#test">test suite</a> will automatically set <code>*USE-BMH-MATCHERS*</code> to <code>NIL</code> while you're running the default test. <p> -Note: Due to the nature of <code>LOAD-TIME-VALUE</code> and the <a -href="#compiler-macro">compiler macro for <code>SCAN</code></a> some +Note: Due to the nature of <a href="http://www.lispworks.com/documentation/HyperSpec/Body/s_ld_tim.htm"><code>LOAD-TIME-VALUE</code></a> and the <a +href="#compiler-macro">compiler macro for <code>SCAN</code> and other functions</a>, some scanners might be created in a <a href="http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_n.htm#null_lexical_environment">null lexical environment</a> at load time or at compile time so be careful @@ -1298,7 +1370,7 @@ time.</blockquote> <br><a class=none name="*allow-quoting*"><b>*allow-quoting*</b></a> <blockquote><br> -If this value is <em>true</em> (the default is <code>NIL</code>) +If this value is <em>true</em> (the default is <code>NIL</code>), CL-PPCRE will support <code>\Q</code> and <code>\E</code> in regex strings to quote (disable) metacharacters. Note that this entails a slight performance penalty when creating scanners because (a copy of) the regex @@ -1312,14 +1384,15 @@ about the converted string and not about the original regex string. NIL * (let ((cl-ppcre:*allow-quoting* t)) - (cl-ppcre:scan "^\\Qa+\\E$" "a+")) + <font color=orange>;;we use CREATE-SCANNER because of Lisps like SBCL that don't have an interpreter</font> + (cl-ppcre:scan (cl-ppcre:create-scanner "^\\Qa+\\E$") "a+")) 0 2 #() #() * (let ((cl-ppcre:*allow-quoting* t)) - (cl-ppcre:scan "\\Qa()\\E(?#comment\\Q)a**b" "()ab")) + (cl-ppcre:scan (cl-ppcre:create-scanner "\\Qa()\\E(?#comment\\Q)a**b") "()ab")) Quantifier '*' not allowed at position 19 in string "a\\(\\)(?#commentQ)a**b" </pre> @@ -1341,9 +1414,135 @@ understand (and Lisp-ier) if you write it like this: </pre> Make sure you also read <a href="#quote">the relevant section</a> in "<a href="#bugs">Bugs and problems</a>." +<p> +Note: Due to the nature of <a href="http://www.lispworks.com/documentation/HyperSpec/Body/s_ld_tim.htm"><code>LOAD-TIME-VALUE</code></a> and the <a +href="#compiler-macro">compiler macro for <code>SCAN</code> and other functions</a>, some +scanners might be created in a <a +href="http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_n.htm#null_lexical_environment">null +lexical environment</a> at load time or at compile time so be careful +to which value <code>*ALLOW-QUOTING*</code> is bound at that +time.</blockquote> </blockquote> +<p><br>[Special variable] +<br><a class=none name="*allow-named-registers*"><b>*allow-named-registers*</b></a> + +<blockquote><br> +If this value is <em>true</em> (the default is <code>NIL</code>), +CL-PPCRE will support <code>(?<i><name>"<regex>"</i>)</code> and <code>\k<i><name></i></code> in regex +strings to provide named registers and back-references as in <a href="http://www.franz.com/support/documentation/7.0/doc/regexp.htm#regexp-new-capturing-2">AllegroCL</a>. <code><i>name</i></code> is has to start with a letter and can contain only alphanumeric characters or minus sign. Names of registers are matched case-sensitively. +The <a href="#create-scanner2">parse tree syntax</a> is not affected by the <code>*ALLOW-NAMED-REGISTERS*</code> switch, <code>:NAMED-REGISTER</code> and <code>:BACK-REFERENCE</code> forms are always resolved as expected. There are also no restrictions on register names in this syntax except that they have to be strings. + +<p> +Examples: + +<pre> +<font color=orange>;; Perl compatible mode (*ALLOW-NAMED-REGISTERS* is NIL)</font> +* (cl-ppcre:create-scanner "(?<reg>.*)") +Character 'r' may not follow '(?<' at position 3 in string "(?<reg>)" + +<font color=orange>;; just unescapes "\\k"</font> +* (cl-ppcre::parse-string "\\k<reg>") +"k<reg>" +</pre> + +<pre> +* (setq cl-ppcre:*allow-named-registers* t) +T + +* (cl-ppcre:create-scanner "((?<small>[a-z]*)(?<big>[A-Z]*))") +#<CLOSURE (LAMBDA (STRING CL-PPCRE::START CL-PPCRE::END)) {AD75BFD}> +(NIL "small" "big") + +<font color=orange>;; the scanner doesn't capture any information about named groups - +;; you have to store the second value returned from CREATE-SCANNER yourself</font> +* (cl-ppcre:scan * "aaaBBB") +0 +6 +#(0 0 3) +#(6 3 6) +</pre> + +<pre> +<font color=orange>;; parse tree syntax</font> +* (cl-ppcre::parse-string "((?<small>[a-z]*)(?<big>[A-Z]*))") +(:REGISTER + (:SEQUENCE + (:NAMED-REGISTER "small" + (:GREEDY-REPETITION 0 NIL (:CHAR-CLASS (:RANGE #\a #\z)))) + (:NAMED-REGISTER "big" + (:GREEDY-REPETITION 0 NIL (:CHAR-CLASS (:RANGE #\A #\Z)))))) + +* (cl-ppcre:create-scanner *) +#<CLOSURE (LAMBDA (STRING CL-PPCRE::START CL-PPCRE::END)) {B158E3D}> +(NIL "small" "big") +</pre> + +<pre> +<font color=orange>;; multiple-choice back-reference</font> +* (cl-ppcre:scan "^(?<reg>[ab])(?<reg>[12])\\k<reg>\\k<reg>$" "a1aa") +0 +4 +#(0 1) +#(1 2) + +* (cl-ppcre:scan "^(?<reg>[ab])(?<reg>[12])\\k<reg>\\k<reg>$" "a22a") +0 +4 +#(0 1) +#(1 2) +</pre> + + +<pre> +<font color=orange>;; demonstrating most-recently-seen-register-first property of back-reference; +;; "greedy" regex (analogous to "aa?")</font> +* (cl-ppcre:scan "^(?<reg>)(?<reg>a)(\\k<reg>)" "a") +0 +1 +#(0 0 1) +#(0 1 1) + +* (cl-ppcre:scan "^(?<reg>)(?<reg>a)(\\k<reg>)" "aa") +0 +2 +#(0 0 1) +#(0 1 2) +</pre> + +<pre> +<font color=orange>;; switched groups +;; "lazy" regex (analogous to "aa??")</font> +* (cl-ppcre:scan "^(?<reg>a)(?<reg>)(\\k<reg>)" "a") +0 +1 +#(0 1 1) +#(1 1 1) + +<font color=orange>;; scanner ignores the second "a"</font> +* (cl-ppcre:scan "^(?<reg>a)(?<reg>)(\\k<reg>)" "aa") +0 +1 +#(0 1 1) +#(1 1 1) + +<font color=orange>;; "aa" will be matched only when forced by adding "$" at the end</font> +* (cl-ppcre:scan "^(?<reg>a)(?<reg>)(\\k<reg>)$" "aa") +0 +2 +#(0 1 1) +#(1 1 2) +</pre> +Note: Due to the nature of <a href="http://www.lispworks.com/documentation/HyperSpec/Body/s_ld_tim.htm"><code>LOAD-TIME-VALUE</code></a> and the <a +href="#compiler-macro">compiler macro for <code>SCAN</code> and other functions</a>, some +scanners might be created in a <a +href="http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_n.htm#null_lexical_environment">null +lexical environment</a> at load time or at compile time so be careful +to which value <code>*ALLOW-NAMED-REGISTERS*</code> is bound at that +time.</blockquote> +</blockquote> + <p><br>[Function] <br><a class=none name="quote-meta-chars"><b>quote-meta-chars</b> <i>string</i> => <i>string'</i></a> @@ -1392,8 +1591,8 @@ string or to convert a parse tree into its internal representation. This is a direct subtype of <a href="#ppcre-error"><code>PPCRE-ERROR</code></a> with two additional slots. These denote the regex string which HTML-PPCRE was parsing and -the position within the string where the error occured. If the error -happens while CL-PPCRE is converting a parse tree both of these slots +the position within the string where the error occurred. If the error +happens while CL-PPCRE is converting a parse tree, both of these slots contain <code>NIL</code>. (See the next two entries on how to access these slots.) <p> @@ -1420,11 +1619,11 @@ The last message we received was "Quantifier '*' not allowed". </blockquote> <p><br>[Function] -<br><a class=none name="ppcre-syntax-error-string"><b>ppcre-syntax-error-string</b></a> <i>condition</i> => <i>string</i></a> +<br><a class=none name="ppcre-syntax-error-string"><b>ppcre-syntax-error-string</b></a> <i>condition</i> => <i>string</i> <blockquote><br> If <code><i>condition</i></code> is a condition of type <a -href="#ppcre-syntax-error"><code>PPCRE-SYNTAX-ERROR</code></a> this +href="#ppcre-syntax-error"><code>PPCRE-SYNTAX-ERROR</code></a>, this function will return the string the parser was parsing when the error was encountered (or <code>NIL</code> if the error happened while trying to convert a parse tree). This might be particularly useful when <a @@ -1434,13 +1633,13 @@ href="#create-scanner"><code>CREATE-SCANNER</code></a> function. </blockquote> <p><br>[Function] -<br><a class=none name="ppcre-syntax-error-pos"><b>ppcre-syntax-error-pos</b></a> <i>condition</i> => <i>number</i></a> +<br><a class=none name="ppcre-syntax-error-pos"><b>ppcre-syntax-error-pos</b></a> <i>condition</i> => <i>number</i> <blockquote><br> If <code><i>condition</i></code> is a condition of type <a -href="#ppcre-syntax-error"><code>PPCRE-SYNTAX-ERROR</code></a> this +href="#ppcre-syntax-error"><code>PPCRE-SYNTAX-ERROR</code></a>, this function will return the position within the string where the error -occured (or <code>NIL</code> if the error happened while trying to +occurred (or <code>NIL</code> if the error happened while trying to convert a parse tree). </blockquote> @@ -1454,13 +1653,10 @@ regex building blocks. Filters can only be used within <a href="#create-scanner2">parse trees</a>, not within Perl regex strings. <p> -Note that filters are currently considered an experimental feature and -their API might change in the future. -<p> A filter is defined by its <em>filter function</em> which must be a function of one argument. During the parsing process this function might be called once or several times or it might not be called at -all. If it's called its argument is an integer <code><i>pos</i></code> +all. If it's called, its argument is an integer <code><i>pos</i></code> which is the current position within the target string. The filter can either return <code>NIL</code> (which means that the subexpression represented by this filter didn't match) or an integer not smaller @@ -1470,7 +1666,7 @@ wants to consume <code>N</code> characters should return <code>(+ POS N)</code>. <p> If you supply the optional value <code><i>length</i></code> and it is -not <code>NIL</code> then this is a promise to the regex engine that +not <code>NIL</code>, then this is a promise to the regex engine that your filter will <em>always</em> consume <em>exactly</em> <code><i>length</i></code> characters. The regex engine might use this information for optimization purposes but it is otherwise irrelevant @@ -1499,7 +1695,7 @@ value of this variable is <code>NIL</code>. <li><CODE>CL-PPCRE::*REG-STARTS*</CODE> and <CODE>CL-PPCRE::*REG-ENDS*</CODE>: Two simple vectors which denote the start and end indices of registers within the regular expression. The -first register is indexed by 0. If a register hasn't matched yet +first register is indexed by 0. If a register hasn't matched yet, then its corresponding entry in <CODE>CL-PPCRE::*REG-STARTS*</CODE> is <code>NIL</code>. @@ -1568,7 +1764,7 @@ NIL * (defun my-weird-filter (pos) "Only match at this point if either pos is odd and the character - we're looking at is lowerrcase or if pos is even and the next two + we're looking at is lowercase or if pos is even and the next two characters we're looking at are uppercase. Consume these characters if there's a match." (format t "Trying at position ~A~%" pos) @@ -1653,10 +1849,10 @@ NIL * (cl-ppcre-test:test) <font color=orange>;; .... -;; (a list of <a class=noborder href="#perl">incompatibilities with Perl</a>)</font color=orange> +;; (a list of <a class=noborder href="#perl">incompatibilities with Perl</a>)</font> </pre> -(If you're not using MK:DEFSYSTEM or asdf it suffices to build +(If you're not using MK:DEFSYSTEM or asdf, it suffices to build CL-PPCRE and then compile and load the file <code>ppcre-tests.lisp</code>.) <p> @@ -1746,7 +1942,7 @@ matching non-ASCII characters. The <a href="">CL-PPCRE test suite</a> can also be used for benchmarking purposes: If you call <code>perltest.pl</code> with a -command line argument it will be interpreted as the minimum number of seconds +command line argument, it will be interpreted as the minimum number of seconds each test should run. Perl will time its tests accordingly and create output which, when fed to <code>CL-PPCRE-TEST:TEST</code>, will result in a benchmark. Here's an example: @@ -1789,7 +1985,7 @@ NIL We gave two test cases to <code>perltest.pl</code> and asked it to repeat those tests often enough so that it takes at least 0.5 seconds to run each of them. In both cases, CMUCL was about twice as fast as Perl. <p> -Here are some more benchmarks (done with Perl 5.6.1 and CMUCL 18d+): +Here are some more benchmarks (done with Perl 5.6.1 and CMUCL 18d+ in 2002): <p> <table border=1> @@ -1898,8 +2094,8 @@ to <a href="#scan"><code>SCAN</code></a>, <a href="#scan-to-strings"><code>SCAN- <a href="#regex-replace"><code>REGEX-REPLACE</code></a>, or <a href="#regex-replace-all"><code>REGEX-REPLACE-ALL</code></a> is a <a href="http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_c.htm#constant_form">constant form</a>. (But see the notes for <a -href="#regex-char-code-limit"><code>*REGEX-CHAR-CODE-LIMIT*</code></a> and -<a href="#use-bmh-matchers"><code>*USE-BMH-MATCHERS*</code></a>.) +href="#*regex-char-code-limit*"><code>*REGEX-CHAR-CODE-LIMIT*</code></a> and +<a href="#*use-bmh-matchers*"><code>*USE-BMH-MATCHERS*</code></a>.) <p> Here's an example of its effect @@ -2012,9 +2208,9 @@ Another thing to consider is that, for performance reasons, CL-PPCRE assumes that most of the target strings you're trying to match are <a href="http://www.lispworks.com/documentation/HyperSpec/Body/t_smp_st.htm">simple strings</a> and coerces non-simple strings to simple strings before -scanning them. If you plan on working with non-simple strings mostly +scanning them. If you plan on working with non-simple strings mostly, you might consider modifying the CL-PPCRE source code. This is easy: -Change all occurences of <code>SCHAR</code> to <code>CHAR</code> and +Change all occurrences of <code>SCHAR</code> to <code>CHAR</code> and redefine the macro in <code>util.lisp</code> where the coercion takes place - that's all. @@ -2157,7 +2353,7 @@ print 1 if '\E*\E*' =~ /(?:\Q$a\E){2}/; </pre> -If you try to do something similar in CL-PPCRE you get an error: +If you try to do something similar in CL-PPCRE, you get an error: <pre> * (let ((cl-ppcre:*allow-quoting* t) @@ -2179,7 +2375,7 @@ the scary details. It <em>can</em> happen in CL-PPCRE, though. Bummer! <p> What gives? <code>"\\Q...\\E"</code> in CL-PPCRE should only -be used in literal strings. If you want to quote arbitrary strings +be used in literal strings. If you want to quote arbitrary strings, try <a href="http://weitz.de/cl-interpol/">CL-INTERPOL</a> or use <a href="#quote-meta-chars"><code>QUOTE-META-CHARS</code></a>: <pre> @@ -2270,7 +2466,6 @@ differences (most of which probably don't matter very often): href="#parse-tree-synonym">parse tree synonyms</a> and <a href="#filters">filters</a>. <li>The AllegroCL engine <a href="http://www.franz.com/support/documentation/7.0/doc/regexp.htm#regexp-new-compatibility-2">will choke on some regular expressions involving curly braces</a> that are accepted by Perl and CL-PPCRE's native engine. <li>The AllegroCL engine's case-folding mode switch (which is used instead of CL-PPCRE's <a href="#create-scanner"><code>:CASE-INSENSITIVE</code> keyword parameter</a>) <a href="http://www.franz.com/support/documentation/7.0/doc/regexp.htm#regexp-new-matching-2">is currently only effective for ASCII characters</a>. -<li>CL-PPCRE's engine doesn't understand the <a href="http://www.franz.com/support/documentation/7.0/doc/regexp.htm#regexp-new-capturing-2">named register groups</a> provided by AllegroCL. <li>The AllegroCL engine <a href="http://www.franz.com/support/documentation/7.0/doc/regexp.htm#regexp-new-compatibility-2">doesn't support</a> <a href="#*allow-quoting*">quoting of metacharacters</a>. <li>In AllegroCL compatibility mode compiled regular expressions (as returned by <a href="#create-scanner"><code>CREATE-SCANNER</code></a>) aren't functions but structures. </ul> @@ -2294,23 +2489,26 @@ href="http://www.cons.org/cmucl/support.html">mailing list</a> as well as the output of Perl's <code>use re "debug"</code> pragma have been very helpful in optimizing the scanners created by CL-PPCRE. -<p> -The asdf system definitions were kindly provided by Marco +<p> The asdf system definitions were kindly provided by Marco Baringer. Hannu Koivisto provided patches to make the <code>.system</code> files more usable. Thanks to Kevin Rosenberg and Douglas Crosher for pointing out how to be friendly to case-sensitive ACL images. Thanks to Karsten Poeck and JP Massar for their help in making CL-PPCRE work with Corman Lisp. JP Massar and Kent M. Pitman -also helped to improve/fix the test suite and the compiler macro. +also helped to improve/fix the test suite and the compiler macro. <a +href="http://random-state.net/">Nikodemus Siivola</a> provided the +fast charset implementation in <code>charset.lisp</code>. See the <a +href="http://weitz.de/cl-ppcre/CHANGELOG">ChangeLog</a> for several +other people who helped with bug reports or patches. <p> -Thanks to the guys at "Café Olé" in Hamburg +Thanks to the guys at "<a href="http://www.weinhandel-ottensen.de/">Café Olé</a>" in Hamburg where I wrote most of the code and thanks to my wife for lending me her PowerBook to test CL-PPCRE with MCL and OpenMCL. <p> -$Header: /usr/local/cvsrep/cl-ppcre/doc/index.html,v 1.131 2005/11/01 09:51:02 edi Exp $ +$Header: /usr/local/cvsrep/cl-ppcre/doc/index.html,v 1.171 2008/07/03 10:06:17 edi Exp $ <p><a href="http://weitz.de/index.html">BACK TO MY HOMEPAGE</a> </body> -</html>
\ No newline at end of file +</html> |