The KernelFiles class is the main dispatcher which parses each
source file.
In preparation for letting kerneldoc Sphinx extension to import
Python libraries, move regex ancillary classes to a separate
file.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/80bc855e128a9ff0a11df5afe9ba71775dfc9a0f.1744106241.git.mchehab+huawei@kernel.org
Handing nested parenthesis with regular expressions is not an
easy task. It is even harder with Python's re module, as it
has a limited subset of regular expressions, missing more
advanced features.
We might use instead Python regex module, but still the
regular expressions are very hard to understand. So, instead,
add a logic to properly match delimiters.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/74dee485f70b7ce85e90496bfdd360283a677a58.1744106241.git.mchehab+huawei@kernel.org
While doing the conversion, we opted to skip empty sections
(description, return), but this makes harder to see the differences
between kernel-doc (Perl) and kernel-doc.py.
Also, the logic doesn't always work properly. So, change the
way this is done by adding an extra step to remove such
sections, doing it only for Return and Description.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/1b057092a48ba61d92a411f4f6d505b802913785.1744106241.git.mchehab+huawei@kernel.org
Maintaining kernel-doc has been a challenge, as there aren't many
perl developers among maintainers. Also, the logic there is too
complex. Having lots of global variables and using pure functions
doesn't help.
Rewrite the script in Python, placing most global variables
inside classes. This should help maintaining the script in long
term.
It also allows a better integration with kernel-doc Sphinx
extension in the future.
I opted to keep this version as close as possible to what we
have already in Perl. There are some differences though:
1. There is one regular expression that required a rewrite:
/\bSTRUCT_GROUP(\(((?:(?>[^)(]+)|(?1))*)\))[^;]*;/
As this one uses two features that aren't available by the native
Python regular expression module (re):
- recursive patterns: ?1
- atomic grouping (?>...)
Rewrite it to use a much simpler regular expression:
/\bSTRUCT_GROUP\(([^\)]+)\)[^;]*;/
Extra care should be taken when validating this script, as such
replacement might cause some regressions.
2. The filters are now applied only during output generation.
In particular, "nosymbol" argument is only handled there.
It means that, if the same file is processed twice for
different symbols, the warnings will be duplicated.
I opted to use this behavior as it allows the Sphinx extension
to read the file(s) only once, and apply the filtering only
when producing the ReST output. This hopefully will help
to speed up doc generation
3. This version can handle multiple files and multiple directories.
So, if one just wants to produce a big output with everything
inside a file, this could be done with
$ time ./scripts/kernel-doc.py -man . 2>/dev/null >new
real 0m54.592s
user 0m53.345s
sys 0m0.997s
4. I tried to replicate as much as possible the same arguments
from kernel-doc, with about the same behavior, for the
command line parameters starting with a single dash (-parameter).
I also added one letter aliases for each parameter, and a
--parameter (sometimes with a better name).
5. There are some sutile nuances between how Perl handles
certain regular expressions. In special, the qr operatior,
which compiles a regular expression also works as a
non-capturing group. It means that some regexes like
this one:
my $type1 = qr{[\w\s]+};
needs to be mapped as:
type1 = r'(?:[\w\s]+)?'
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/2fa671a9fb08d03a376a42d46cc0b1d3aab4ae3f.1744106241.git.mchehab+huawei@kernel.org
