A long-standing problem with IWYU has been the preprocessor
metaprogramming techniques used in Boost.
IWYU is fine with cycles in the include graph, but did not accept cycles
in the corresponding mapping graph (a potential rationale might have
been to let people know if they explicitly added contradictory mappings,
but we really don't know what drove this decision).
Looking at the algorithm expanding the transitive closure of the mapping
graph, it expands all nodes in depth-first order. Cycle detection is
necessary to protect the algorithm from infinite recursion in the
presence of a cycle. But there should be no difference in the transitive
closure between a graph with cycles and one without. DFS will still
visit all nodes, it will just avoid visiting the same subtree twice.
Once the expansion is done, mapping lookup is an inherently
non-recursive operation (made cheaper by performing the recursive
expansion up-front). See the GetCandidateHeadersForFilepath family of
functions and their respective callers.
Two conclusions:
* Cycles are already detected by the transitive closure algorithm
* No code _after_ the transitive closure is affected by any cycles
Therefore, update the transitive closure expansion to ignore cycles
instead of failing on them. Add diagnostic logging at level 8 so we can
look into its decision-making if necessary.
Fixes issue #424
Non-sugared template specialization type template arguments are already
analyzed due to 'TraverseType' call inside
'InstantiatedTemplateVisitor::TraverseSubstTemplateTypeParmTypeHelper'
and subsequent 'TraverseTemplateSpecializationTypeHelper' call.
Components of sugared template specialization types are added into
resugar_map, otherwise their template arguments would not be reported.
Test case has been extended.
Full template specialization type use requires full information about
its template-argument-dependent fields and nested typedefs. But earlier,
it wasn't reported in many cases when template specialization type isn't
written explicitly in non-fwd-decl context.
Installation of libclang-16-dev would fail with:
Unpacking libclang-16-dev (1:16~++20221216031545+6c5f3f62bdb2-1~exp1~20221216151636.624) ...
dpkg: error processing archive /var/cache/apt/archives/libclang-16-dev_1%3a16~++20221216031545+6c5f3f62bdb2-1~exp1~20221216151636.624_amd64.deb (--unpack):
trying to overwrite '/usr/lib/x86_64-linux-gnu/libclang-16.so.1', which is also in package libclang1-16 1:16~++20221216031545+6c5f3f62bdb2-1~exp1~20221216151636.624
...
Unpacking libclang-dev (1:16.0-57~exp1~20221014130100.5) ...
Errors were encountered while processing:
/var/cache/apt/archives/libclang-16-dev_1%3a16~++20221216031545+6c5f3f62bdb2-1~exp1~20221216151636.624_amd64.deb
...
E: Sub-process /usr/bin/dpkg returned an error code (1)
Error: Process completed with exit code 100.
libclang-16-dev somehow depends on libclang1-16, and both are trying to
install the same libclang1.so.
Use --force-override switch to comfort dpkg, even though it's not
recommended in practice. See e.g. https://askubuntu.com/a/491086/852068.
Let's remove this as soon as possible.
The libclang-dev packages are actually called libclang-NN-dev, not
libclang-dev-NN, so make the glob a little more permissive to remove
more preinstalled packages.
The `python` command may not exist on a system with Python 3 installed.
See https://peps.python.org/pep-0394/ for a discussion of the commands
expected to be installed.
Since Python2 was officially sunsetted almost three years ago
(https://www.python.org/doc/sunset-python-2/), IWYU should prefer
compatibility with newer systems over older ones.
Python code in scripts is still Python3/Python2-compatible, but we will no
longer make an effort to preserve Python2 support over time.
Fixes#1096.
Commit 04f1c92537 added a mechanism that
dynamically adds a mapping from the includee to the includer in this
case:
// a.h
#if CONTROL_FLAG
void foo();
#else
void bar();
#endif
// b.h
#define CONTROL_FLAG
#include "a.h"
However, there was no check that includee and includer were different,
so regular #include guards would cause self-mappings, which would
eventually lead to cycles in mappings and assertion failures, e.g.:
Cycle in include-mapping:
<z.hpp> ->
<z.hpp>
,,,/iwyu_include_picker.cc:845: Assertion failed: Cycle in include-mapping
Aborted
Avoid this by checking if the file defining the macro is the same as the
file using it; in that case we don't need a mapping.
* Do all this logging on level 8, whether logging is on for file or not
* Always say "Add dynamic mappings..."
* Don't log quoted from->to, that's already logged in AddMapping
* State the reason dynamic mapping is added
* Include file paths where available
This should make it easier to troubleshoot strange effects from dynamic
mappings triggered by IWYU itself.
Only attempt to report decl uses of template names that have an
underlying template decl. This ignores uses of dependent templates and
potentially other such variations.
Even if that causes us to miss reporting for some valid constructs, it
allows further analysis of files that would otherwise crash IWYU.
Fixes issue #1140
Add using-declaration for clang::TemplateDecl, and remove explicit
namespace qualifiers for use of RecordDecl and CXXRecordDecl, which are
already pulled in with using-declarations.
No functional change intended.
Boolean type is internally represented in clang as '_Bool' builtin type.
In order to display it correctly taking into account language options,
actual printing policy should be used.
The clang-resource-headers CMake target is not exported from the
packaged LLVM/Clang CMake modules, so we synthesize it by explicitly
copying resource/built-in headers from the package install root to the
build dir.
Clang 16 changed the structure not to use major.minor.patch but rather
just the major version (in line with LLVM versioning at large).
Adjust so we pick up the headers from the right place, and put them
where Clang libraries will be looking.
this was a proposed issue in #1095 where one could
make a block of keeps as opposed to marking every
keep with a pragma: keep instruction.
added test to verify and updated documentation
where appropriate.
clang 1acffe81ee9117691812b9bf8747c03354177d15 changed the API for
clang::TemplateSpecializationType to no longer have getArgs() or getNumArgs()
This change replaces them with template_args() and retains the previous logic
This avoid double-anchoring, for example if users put their own anchors
in mappings. It also reduces clutter in the RegexMatch/RegexReplace
functions now that both need to do the same anchoring.
This is useful to generically remap files from one directory to another.
The following mapping would for instance map all files under the "foo/"
directory to "third_party/foo/", for instance, mapping "foo/bar.h" to
"third_party/foo/bar.h".
[ { include: ['@"(foo/.*)"', private, '"third_party/\1"', public ] } ]
This is useful for large project where different third_party libraries
are compiled together. In Chromium for instance, include paths would
look like:
-Ithird_party -Ithird_party/v8/include
This allows code in V8 to include its own headers directly. Other
third_parties would include them via "third_party/v8/include/...".
Because files are accessible from two different paths, IWYU can't know
which should be used. Using a mapping file, adding or removing the
"third_party/v8/include" prefix where needed, resolves this problem.
This change provides better location info when using macros such as:
BOOST_CHECK_THROW(..., Exc);
The use of type Exc would be attributed to the file defining the macro
before this patch, but is now correctly attributed to the expansion
location.
Co-authored-by: Kim Gräsman <kim.grasman@gmail.com>
A function may just transmit passed-by-reference parameter somewhere.
Requirement to explicitly write forward declaration in the same file
(.cpp-file) to avoid '#include' suggestion is impractical when that type
is already fwd-declared in the corresponding header.
'Autocast' may still make sense for header-defined functions, due to
unlimited number of possible callers, so analysis of those fuctions
is kept.
Both function declaration site handling and call site handling
are changed.
Running the tests/cxx/comment_pragmas.cc test on Windows fails saying
that <some_system_header_file> is not diagnosed correctly.
Turns out to be because our custom lexer for GNU @headername comment
directives did not respect Windows line endings, and failed properly
identify the @headername.
Match on both CR and LF when attempting to find end of line.
As a fix for issue #247 we added a block of hard-coded paths to the
include search path, which would find libc++ from the MacOS SDK.
While the MacOS SDK is usually present where IWYU is run, on developer
machines, there may be any number of other sysroots providing libc++
that users would prefer to use.
The Nix tool is one such player, where builds are isolated and all
dependencies are gathered in a single root for reproducibility.
Recent experiments with Clang on different Mac systems show that:
* if the MacOS SDK is present, IWYU picks it up automatically via the
Clang driver
* if the MacOS SDK is not present, no include paths are conjured out of
thin air
* Clang installed from Homebrew llvm package automatically finds the
libc++ also provided as part of that package
It seems only IWYU forcefully adds magic paths, and it causes
problems for folks who want to do the right thing and build the sysroot
themselves. So remove these hard-coded paths.
If there is no installation of libc++ discoverable to Clang's probing,
or an alternative libc++ is desired, users can disable probing and
specify relevant paths directly on command-line:
# for Homebrew version of llvm-14
include-what-you-use -nostdinc++ \
-isystem /opt/homebrew/opt/llvm\@14/include/c++/v1 \
sourcefile.cc
# for Apple Command Line Tools version of libc++
include-what-you-use -nostdinc++ \
-isystem /Library/Developer/CommandLineTools/usr/include/c++/v1 \
sourcefile.cc
The mechanics of this technique is described in more detail for Clang:
https://libcxx.llvm.org//UsingLibcxx.html#using-a-custom-built-libc.
Co-authored-by: Kim Gräsman <kim.grasman@gmail.com>
Some standard libraries have a [[nodiscard]] attribute on std::find,
which leads to a warning about unused return value, which in turns masks
any real results from the test.
Avoid std::ignore since this test is older than C++11.
It is needed in order to be able to add definition to passing-by-ref
function and test on it "autocast" rules, not type info requirement
due to parameter variable definition.
New 'DirectStruct*'s and 'TplDirectStruct*'s are added because
requirement to '#include' their header is tested only by that
'#include', no inline diagnostics are emitted.
Clang recently moved to gnu++17 by default, where dynamic exception
specifications are no longer supported. Upstream change:
3e99b8d947
Since badinc.cc is all pre-C++11 code, pin the language standard for
it to gnu++98, which has a number of interesting benefits:
* makes it less rewarding to add new tests here
* any standard library changes have significantly lower impact
* it's about 30% faster
No functional change.
This little beauty would cause an assertion failure in GetCanonicalName:
printf "#include <stdio.h>" | include-what-you-use -c -x c -
Special-case '<stdin>' like we do with '<built-in>'.
Fixes issue #1105.
There is no need in reporting enum opaque declaration or full use if
enumeration type name or enumerator name isn't explicitly written,
because it should already be present elsewhere in the translation unit,
probably through included headers.
Only LLVM regexes need explicit anchoring; std::regex_match already has
full match semantics.
This saves us thousands of temporaries when using std::regex-based
dialects and actually improves processing time of
$ time include-what-you-use -Xiwyu --regex=ecmascript -Xiwyu --mapping_file=qt5_11.imp tests/981/t.cc
...
real 0m5,517s
user 0m5,504s
sys 0m0,013s
if only very slightly (~10%).
When building regexes from @-prefixed mappings, we wrapped the
expression in ^(...)$ to force llvm::Regex::match to match only the
whole string.
The parentheses were probably added in there by accident, we have no use
for a capture group in this scenario, and the anchors are fine without
grouping.
As reported in issue #981, using std::regex in IWYU has caused a
tremendous performance regression for large mapping files containing
regex mappings.
$ cat t.cc
#include <string>
# with llvm::Regex
$ time include-what-you-use -Xiwyu --mapping_file=qt5_11.imp t.cc
...
real 0m0,529s
user 0m0,509s
sys 0m0,020s
# with std::regex
$ time include-what-you-use -Xiwyu --mapping_file=qt5_11.imp t.cc
...
real 0m29,870s
user 0m29,717s
sys 0m0,012s
qt5_11.imp contains 2300+ regex mappings, and <string> has a bunch of
includes, so this is a good testbed for regular expression engines, but
over 50x slower is not the result we were hoping for.
The reason we switched to std::regex was to get support for negative
lookaround (llvm::Regex does not have it), but exotic regexes in
mappings are pretty rare, and this is a significant performance hit.
Introduce a --regex option to select regex dialect, with documented
tradeoffs. Put the default back to LLVM's fast implementation.
This fixes issue #981.
Kim Gräsman