C/C++ Dependency Analyzer: a rewrite of John Lakos' dep_utils (adep/cdep/ldep) from

Overview

logo

https://travis-ci.org/rakhimov/cppdep.svg?branch=master 'Build status' Code Health

cppdep performs dependency analysis among components/packages/package groups of a large C/C++ project. This is a rewrite of dep_utils(adep/cdep/ldep), which is provided by John Lakos' book "Large-Scale C++ Software Design", Addison Wesley (1996).

Limitations

  • Indirect extern declarations of global variables or functions instead of including the proper component header with the declarations.
  • Embedded dynamic dependencies, such as dynamic loading and configurable internal services.
  • Preprocessing or macro expansion is not performed. Dependency inclusion via preprocessor meta-programming is not handled.
  • Dependency exclusion with C style multi-line comments or macros is not respected.

Requirements

  1. Python 2.7 or 3.4+
  2. NetworkX
  3. pydot
  4. pydotplus
  5. PyYAML
  6. PyKwalify 1.6.0+

The dependencies can be installed with pip.

$ sudo pip install -r requirements.txt

Installation

From the source:

$ ./setup.py install

The latest stable release from PyPi:

$ pip install cppdep

Usage

Create a configuration file that describes the project for analysis. config_schema.yml is given for guidance.

In the root directory of the project with the configuration file, run the following command to generate dependency analysis reports and graphs.

$ cppdep -c /path/to/config/file

More documentation and example configurations can be found in project wiki.

Acknowledgments

  • John Lakos for inventing the analysis and providing dep_utils.
  • Zhichang Yu for rewriting dep_utils into Python.
Comments
  • Behavior specification for anomalous conflicting component files

    Behavior specification for anomalous conflicting component files

    Anomalous (rare, error) case of having "component.c" and "component.cc" or "component.h" and "component.hpp" at the same time trying to define the same component. In other words, files with the same basenames but different extensions in header and/or source groups.

    opened by rakhimov 1
  • Missing 'pydot' dependency fails on networkx nx_pydot

    Missing 'pydot' dependency fails on networkx nx_pydot

    Traceback (most recent call last):
      File "/.../python3env/bin/cppdep", line 11, in <module>
        sys.exit(main())
      File "/.../python3env/lib/python3.5/site-packages/cppdep/__main__.py", line 59, in main
        analysis.analyze(printer, args)
      File "/.../python3env/lib/python3.5/site-packages/cppdep/cppdep.py", line 797, in analyze
        lambda x: isinstance(x, PackageGroup)))
      File "/.../python3env/lib/python3.5/site-packages/cppdep/cppdep.py", line 779, in _analyze
        digraph.write_dot(graph_name)
      File "/.../python3env/lib/python3.5/site-packages/cppdep/graph.py", line 273, in write_dot
        write_dot(self.digraph, file_basename + '.dot')
      File "<decorator-gen-416>", line 2, in write_dot
      File "/.../python3env/lib/python3.5/site-packages/networkx/utils/decorators.py", line 224, in _open_file
        result = func(*new_args, **kwargs)
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 54, in write_dot
        P = to_pydot(G)
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 199, in to_pydot
        pydot = _import_pydot()
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 348, in _import_pydot
        import pydot
    
    bug 
    opened by rakhimov 0
  • Windows: Unicode Escape Error NetworkX, pydotplus and Python 2.7

    Windows: Unicode Escape Error NetworkX, pydotplus and Python 2.7

    If component/project names contain paths as identifiers, the path separator on Windows \ followed by 'U' or 'u' is interpreted as a Unicode literal by pydotplus plotting on Windows with python 2.7.

    This problem does not show up on python 3.

    For the sake of consistency in the report (stable report) and testing, use Unix path separator in Ids.

    bug 
    opened by rakhimov 0
  • Handle 'ipp' template implementation/source files

    Handle 'ipp' template implementation/source files

    Even though templates are only in headers, ipp files could be considered as implementation pair of the interface header. This occurs, for example, in Boost. #29.

    opened by rakhimov 0
  • Redundant 'ldep' printing of cumulative dependencies

    Redundant 'ldep' printing of cumulative dependencies

    Unlike the original ldep, the cppdep is printing the whole cumulative dependencies; that is, the all link time dependencies are printed. This information is visually provided in the graph, and implicitly provided in the original ldep '-l | -L' flags.

    It seems like the current implementation of the cumulative dependency printing is geared towards debugging rather than analysis report.

    Fix this printing by implementing the original behavior of ldep.

    bug 
    opened by rakhimov 0
  • Pairing header and implementation files in different locations

    Pairing header and implementation files in different locations

    Header and implementation files can be located in different directories, e.g., headers in include and implementation in src. The current implementation assumes the same location.

    bug enhancement 
    opened by rakhimov 0
  • Deduce external packages from the include directives w/o filesystem search

    Deduce external packages from the include directives w/o filesystem search

    The package can be deduced from the include directives following the pattern: "<package/header>", e.g., "<boost/any.hpp>". This approach will avoid the relatively expensive lookup of the header from the system. Whether such external header actually exists on the system or not is irrelevant to the dependency analysis.

    This heuristic would make the configuration simpler and more robust for cross-platform work.

    In addition, the standard library headers can be "hard-coded" into the script since these are the most likely to be used/searched by the source files.

    There's a small chance of false positives if the project under analysis happens not to conform to this convention and invents its own packages with the same names as external ones, e.g., boost, qt, libxml.

    enhancement 
    opened by rakhimov 0
  • Double counting of common components in CCD

    Double counting of common components in CCD

    The current implementation overcounts link time dependencies with common components deep in the dependency graph. Consider: A->B->C, A->D->C. CCD(A) must be 4 (4 object files in total.) The current approach overcounts the common object C, so CCD(A) ends up being 5.

    bug 
    opened by rakhimov 0
  • Extended definition for 'Component'

    Extended definition for 'Component'

    Even though John Lakos defined a component as a pair of h and c files, C++ can have template only components residing only in header files (e.g., STL/Boost/etc.). Moreover, some header-only components may contain only inline functions or macros without any need for an implmentation file (e.g., inline math, Boost PPL). For these reason, unpaired header files can be counted as components by default.

    In addition, the implementation file containing the main function of an application could be considered as a component as well (an entry point).

    bug enhancement 
    opened by rakhimov 0
  • Incorrect dependency processing with file basenames

    Incorrect dependency processing with file basenames

    The current implementation uses the basename of the included files as the key in dependency search. This approach ignores the current location of the dependent files and the language include rules, i.e. <header.h> vs. "header.h". Moreover, it results in name conflicts for files that may belong to completely separate projects or packages.

    bug 
    opened by rakhimov 0
  • Compatibility fixes

    Compatibility fixes

    This PR contains some fixes that I needed to get the cppdep tool running in my recent python 3.8 environment. Also handling of errors in the yaml file was adapted a little bit to get more helpful error messages

    opened by jsinge 0
  • Missing graph module

    Missing graph module

    I've just installed this project, tried to run it but I get this error message.

    Traceback (most recent call last):
      File "cppdep.py", line 36, in <module>
        from .graph import Graph
    ModuleNotFoundError: No module named '__main__.graph'; '__main__' is not a package
    
    bug help wanted question 
    opened by nbourre 2
  • Python 3.6 os.path.commonprefix expects List

    Python 3.6 os.path.commonprefix expects List

      File "/.../cppdep/cppdep.py", line 92, in path_common
        path = os.path.commonprefix(paths)
      File "/home/olzhas/temp/pyenv/lib/python3.6/genericpath.py", line 76, in commonprefix
        if not isinstance(m[0], (list, tuple)):
    TypeError: 'set' object does not support indexing
    
    bug 
    opened by rakhimov 1
  • Include Wrangler Analysis

    Include Wrangler Analysis

    Incorporation of include-wrangler analysis

    • header file cost
    • include cost
    • translation unit cost
    • other features

    In collaboration with include-wrangler author @lukedodd

    enhancement 
    opened by rakhimov 0
  • Options to disable/enable warnings

    Options to disable/enable warnings

    • [ ] Duplicate include
    • [ ] Redundant (transitive from the component header) include
    • [ ] Missing a component header (incomplete component)
    • [ ] Failure to locate a header from an include directive
    enhancement 
    opened by rakhimov 0
Releases(0.2.0)
  • 0.2.0(Feb 3, 2017)

    Added

    • Pairing header and implementation files in different locations (#19)
    • Handle 'ipp' template implementation source files (#31)
    • Behavior specification for anomalous conflicting component files (#27)
    • Implement ignore/exclude paths (#23)
    • Accept glob pattern for source paths (#36)
    • Project wiki pages
    • Regex pattern based include directive classification (#22)
    • Deduce external packages from the include directive w/o filesystem search (#18)
    • Handle header files w/o extensions (Boost/STL/Qt/etc.) (#32)
    • Use POSIX path separator in component names (for cross-platform report stability)
    • Configuration file validation against the schema (with PyKwalify)

    Changed

    • pytest instead of nose
    • YAML configuration files instead of XML (#24)

    Removed

    • Implicit single-path alias Package construction

    Fixed

    • Exception leaks out of main()
    • Unicode Escape Error on graph dot on Windows with Python 2.7 (#35)
    • Python3 UnicodeDecodeError for 'utf-8' in source files (#30)
    • Logging: Type Error: not all arguments converted during string formatting (#28)
    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Jan 6, 2017)

    Added

    • The original ldep '-l|-L' options to print dependencies (#20)
    • '-o' to print reports into a file
    • Warn about duplicate and redundant includes (#13)
    • Extended definition for 'Component' (#7)
    • PEP-257 conformance (#2)
    • PEP-8 conformance (#1)
    • Python 3 support
    • PyPI package
    • XML configuration example and RNG schema
    • Travis CI (Linux, OS X) and AppVeyor CI (Windows) setups

    Changed

    • Differentiate 'paths' into source, include, and alias.
    • Print warnings to stderr instead of stdout (#12)
    • Report Component levels instead of Graph layers (#9)
    • Refactor the procedural design into the object-oriented design (#4)
    • Change '-f' flag into '-c' flag
    • Replace optparse with argparse
    • XML configuration file format

    Removed

    • Redundant printing a list of cumulative dependencies (#20)
    • Indirect missing-header include warnings
    • Global cross-package and cross-package-group component dependency analysis
    • 'details-of-components/--debug' verbosity
    • dot2any.py helper script
    • Manual profiling code (use pyvmmonitor instead)
    • Manual testing code (automated with nosetest)

    Fixed

    • Level 0 External components missing from the report and graph (#21)
    • Incorrect dependency processing with file basenames (#6)
    • Wrong level calculation for cycles (#8)
    • Double counting of common components in CCD calculations (#11)
    • Missing cycles from the Dot graph (#10)
    • Outdated networkx API usage
    Source code(tar.gz)
    Source code(zip)
Owner
Olzhas Rakhimov
Olzhas Rakhimov
Data parsing and validation using Python type hints

pydantic Data validation and settings management using Python type hinting. Fast and extensible, pydantic plays nicely with your linters/IDE/brain. De

Samuel Colvin 12.1k Jan 05, 2023
A formatter for Python files

YAPF Introduction Most of the current formatters for Python --- e.g., autopep8, and pep8ify --- are made to remove lint errors from code. This has som

Google 13k Dec 31, 2022
An analysis tool for Python that blurs the line between testing and type systems.

CrossHair An analysis tool for Python that blurs the line between testing and type systems. THE LATEST NEWS: Check out the new crosshair cover command

Phillip Schanely 836 Jan 08, 2023
🦔 PostHog is developer-friendly, open-source product analytics.

PostHog provides open-source product analytics, built for developers. Automate the collection of every event on your website or app, with no need to send data to 3rd parties. With just 1 click you ca

PostHog 10.3k Jan 01, 2023
Auto-generate PEP-484 annotations

PyAnnotate: Auto-generate PEP-484 annotations Insert annotations into your source code based on call arguments and return types observed at runtime. F

Dropbox 1.4k Dec 26, 2022
Optional static typing for Python 3 and 2 (PEP 484)

Mypy: Optional Static Typing for Python Got a question? Join us on Gitter! We don't have a mailing list; but we are always happy to answer questions o

Python 14.4k Jan 05, 2023
Inspects Python source files and provides information about type and location of classes, methods etc

prospector About Prospector is a tool to analyse Python code and output information about errors, potential problems, convention violations and comple

Python Code Quality Authority 1.7k Dec 31, 2022
A system for Python that generates static type annotations by collecting runtime types

MonkeyType MonkeyType collects runtime types of function arguments and return values, and can automatically generate stub files or even add draft type

Instagram 4.1k Jan 02, 2023
TidyPy is a tool that encapsulates a number of other static analysis tools and makes it easy to configure, execute, and review their results.

TidyPy Contents Overview Features Usage Docker Configuration Ignoring Issues Included Tools Included Reporters Included Integrations Extending TidyPy

Jason Simeone 33 Nov 27, 2022
Collection of library stubs for Python, with static types

typeshed About Typeshed contains external type annotations for the Python standard library and Python builtins, as well as third party packages as con

Python 3.3k Jan 02, 2023
Python package to parse and generate C/C++ code as context aware preprocessor.

Devana Devana is a python tool that make it easy to parsing, format, transform and generate C++ (or C) code. This tool uses libclang to parse the code

5 Dec 28, 2022
Run-time type checker for Python

This library provides run-time type checking for functions defined with PEP 484 argument (and return) type annotations. Four principal ways to do type

Alex Grönholm 1.1k Dec 19, 2022
Static type checker for Python

Static type checker for Python Speed Pyright is a fast type checker meant for large Python source bases. It can run in a “watch” mode and performs fas

Microsoft 9.4k Jan 07, 2023
Guesslang detects the programming language of a given source code

Detect the programming language of a source code

Y. SOMDA 618 Dec 29, 2022
A bytecode vm written in python.

CHex A bytecode vm written in python. hex command meaning note: the first two hex values of a CHex program are the magic number 0x01 (offset in memory

1 Aug 26, 2022
Print a directory tree structure in your Python code.

directory-structure Print a directory tree structure in your Python code. Download You can simply: pip install directory-structure Or you can also: Cl

Gabriel Stork 45 Dec 19, 2022
An app to show the total number of lines of code written by an user.

Lines of code Have you ever wondered how many lines of code you wrote in github? This tool will calculate it for you! To calculate the total number of

B.Jothin kumar 10 Jan 26, 2022
Alarmer is a tool focus on error reporting for your application.

alarmer Alarmer is a tool focus on error reporting for your application. Installation pip install alarmer Usage It's simple to integrate alarmer in yo

long2ice 20 Jul 03, 2022
Unbearably fast O(1) runtime type-checking in pure Python.

Look for the bare necessities, the simple bare necessities. Forget about your worries and your strife. — The Jungle Book.

1.4k Dec 29, 2022
Turn your Python and Javascript code into DOT flowcharts

Notes from 2017 This is an older project which I am no longer working on. It was built before ES6 existed and before Python 3 had much usage. While it

Scott Rogowski 3k Jan 09, 2023