Using global attributes when building models makes it really hard to share assets between different models. Multiple models have to be stored in separate sources, cannot be processed together and unused assets are still drawn in diagrams.

This MR removes all usages of global/static attributes and requires to explicitly pass elements to a model. Note that only Dataflows have to be passed. See the new tests added for difference in usage.

Also note that previously, the sequence diagram relied on the order in which assets were defined (not dataflows). Now dataflows are used to order assets. It's still not consistent with being able to manually order dataflows but this can be improved in future MRs.

Add a categories field to threats so custom threat libraries can use any threat taxonomy.

Formatted the threats.json file using jq so it has consistent identation.

The following attributes are not used in any threat. Using them in models can create a false sense of security where no new findings would be reported.

• authenticatedWith
• authenticationScheme
• codeType
• handlesCrashes
• handlesInterruptions
• hasWriteAccess
• implementsCommunicationProtocol
• isAdmin
• isSQL
• onAWS
• onRDS
• OS
• providesConfidentiality
• storesLogData
• storesPII
• storesSensitiveData
• tracksExecutionFlow

Its pretty easy for anyone to extend pytm classes and add these back when referencing them in custom threats. I believe that extending pytm this way should be encouraged.

This is controversial but it's easier to remove unused attributes than try to figure out correct description for them. Less is more.

I updated the template to showcase #150 and #154. Also expanded template_engine to support the call operator with methods that return a list and support to call methods within a report utility class. Updated template.md with examples of each.

Updated tests and ensured tests passed. Also updated report test to write output_current.md At some point tests could use this but for now can be used to quickly commit the new expected value when the report changes. Added output_current.mdto.gitignore`

I also added an example in the template.md to get the parents of each Boundary, to see this I updated tm.py with a hierarchy of Boundaries. If we think this is too much for the intro tm.py I can back those changes out.

In my use case, I want to synchronize the findings with an external risk system. To be able to do so, I added these features -

1. Each element can be provided with a uniqueId. The idea is that these are specified when writing the model and thus static.
2. Findings will get a uniqueId which is a combination of the uniqueId and the findings's ID. In this way, these are also static and can be used as the primary key to the external risk system.
3. To make communication easier, the Element's uniqueId can be added to the name and thus become visible on the diagrams. This enables smoother dialog when talking about the output as talking about ABC becomes very precise.

I apologize upfront for these things -

• I only changed the pytm.py file to now. Any changes to README.md etc, I'll await your first feedback.
• I spend considerably time trying to get the value of the TM varStrings in the Element class. I could not find a way, so I added a global variable hack.
• I did not add any tests. Any advice on what tests I should add would be welcome.

To make it possible to import other models into the current model without adding all elements the elements of the model are now stored in the TM object and not the class.

I addition I added some test models to check how they differ between each other and between the current master branch. But the result of the models have to be compared manual, because the UUID and the order often differ even though the resulting report and the diagrams are similar. Maybe it is possible to automate this process more.

Since I can only do manual checking it would be good if someone else checked if this PR is not changing the original behavior.

…nAWS and updated logging threat to use storesSensitiveData

-Added missing getters and setters for Datastore -Corrected typo on Element, onAWS to inAWS -Changed default value for storesSensitiveData to True -Updated a logging threat to use storesSensitiveData

Hi all,

thanks for sharing this nice tool. I just wanted to explore the sample but getting an error

➜  pytm git:(master) ✗ ./tm.py --dfd | dot -Tpng -o sample1.png
2019-03-31 07:24:45.271 dot[27759:12821403] +[__NSCFConstantString length]: unrecognized selector sent to class 0x7fff95b4a8c0
2019-03-31 07:24:45.272 dot[27759:12821403] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '+[__NSCFConstantString length]: unrecognized selector sent to class 0x7fff95b4a8c0'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff3df1743d __exceptionPreprocess + 256
	1   libobjc.A.dylib                     0x00007fff69e25720 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff3df941a5 __CFExceptionProem + 0
	3   CoreFoundation                      0x00007fff3deb6ad0 ___forwarding___ + 1486
	4   CoreFoundation                      0x00007fff3deb6478 _CF_forwarding_prep_0 + 120
	5   CoreFoundation                      0x00007fff3de47f54 CFStringCompareWithOptionsAndLocale + 72
	6   ImageIO                             0x00007fff409b5367 _ZN17IIO_ReaderHandler15readerForUTTypeEPK10__CFString + 53
	7   ImageIO                             0x00007fff4098d527 _ZN14IIOImageSource14extractOptionsEP13IIODictionary + 183
	8   ImageIO                             0x00007fff409ba2e6 _ZN14IIOImageSourceC2EP14CGDataProviderP13IIODictionary + 72
	9   ImageIO                             0x00007fff409ba1bb CGImageSourceCreateWithDataProvider + 172
	10  libgvplugin_quartz.6.dylib          0x0000000107cfcc54 quartz_loadimage_quartz + 224
	11  libgvc.6.dylib                      0x0000000107c59781 gvloadimage + 269
	12  libgvc.6.dylib                      0x0000000107c587e0 gvrender_usershape + 955
	13  libgvc.6.dylib                      0x0000000107c8662e poly_gencode + 2129
	14  libgvc.6.dylib                      0x0000000107c92b7b emit_node + 1030
	15  libgvc.6.dylib                      0x0000000107c91805 emit_graph + 4769
	16  libgvc.6.dylib                      0x0000000107c96d0d gvRenderJobs + 4911
	17  dot                                 0x0000000107c4fd62 main + 697
	18  libdyld.dylib                       0x00007fff6aef3085 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
[1]    27758 done       ./tm.py --dfd |
       27759 abort      dot -Tpng -o sample1.png

I installed graphviz via brew, using macOS 10.14 and Python 3.7.3.

model.py --list-elements shows a list of all elements which can be used in a threat model with pytm.

Why? I often find my self looking up the exact names of the elements and there doc string.

Currently the output looks like this.

Elements:
Actor          -- An entity usually initiating actions
Asset          -- An asset with outgoing or incoming dataflows
Boundary       -- Trust boundary groups elements and data with the same trust level.
Dataflow       -- A data flow from a source to a sink
Datastore      -- An entity storing data
ExternalEntity --
Lambda         -- A lambda function running in a Function-as-a-Service (FaaS) environment
Process        -- An entity processing data
Server         -- An entity processing data
SetOfProcesses --

Atributes:
Action         -- Action taken when validating a threat model.
Classification -- An enumeration.
Data           -- Represents a single piece of data that traverses the system
Lifetime       -- An enumeration.
TLSVersion     -- An enumeration.

I want to have stable references so I can synchronize the findings with an risk management tool and make the model a living document. To do so, I allow includeOrder to be set on any component (Actor etc.) and when that is set and the order is specified (not -1), the name is changed to contain the order and the findings are containing the order as well.

In this version, there is no validation of whether the order is unique. That is up to the person writing the Python to ensure that.

On finding object's UniqueId: When order is present and includeOrder is true on the object, this will be formatted as findingId:order. E.g. if finding is INP01 and order is 123, the value becomes INP01:123."

On object's includeOrder: If True and Order is set (not -1), the displayed name will be formatted as 'order:name'. If you make Order unique, this will give you a stable reference you can use for synchronization etc.

Cloned from master today and running the example reports with: ./tm.py --report docs/template.md | pandoc -f markdown -t html > report.html

Produces the following error: pandoc: (TagClose "script") is not a TagOpen CallStack (from HasCallStack): error, called at src/Text/HTML/TagSoup/Type.hs:128:19 in tgsp-0.14.8-2271f385:Text.HTML.TagSoup.Type

I think the error is with pandocs formatting of the following threat: { "SID": "SC04", "target": ["Server"], "description": "XSS Using Alternate Syntax", "details": "An adversary uses alternate forms of keywords or commands that result in the same action as the primary form but which may not be caught by filters. For example, many keywords are processed in a case insensitive manner. If the site's web filtering algorithm does not convert all tags into a consistent case before the comparison with forbidden keywords it is possible to bypass filters (e.g., incomplete black lists) by using an alternate case structure. For example, the script tag using the alternate forms of Script or ScRiPt may bypass filters where script is the only form tested. Other variants using different syntax representations are also possible as well as using pollution meta-characters or entities that are eventually ignored by the rendering engine. The attack can result in the execution of otherwise prohibited functionality.", "Likelihood Of Attack": "High", "severity": "High", "condition": "target.sanitizesInput is False or target.validatesInput is False or target.encodesOutput is False", "prerequisites": "Target client software must allow scripting such as JavaScript.", "mitigations": "Design: Use browser technologies that do not allow client side scripting.Design: Utilize strict type, character, and encoding enforcementImplementation: Ensure all content that is delivered to client is sanitized against an acceptable content specification.Implementation: Ensure all content coming from the client is using the same encoding; if not, the server-side application must canonicalize the data before applying any filtering.Implementation: Perform input validation for all remote content, including remote and user-generated contentImplementation: Perform output validation for all remote content.Implementation: Disable scripting languages such as JavaScript in browserImplementation: Patching software. There are many attack vectors for XSS on the client side and the server side. Many vulnerabilities are fixed in service packs for browser, web servers, and plug in technologies, staying current on patch release that deal with XSS countermeasures mitigates this.", "example": "In this example, the attacker tries to get a script executed by the victim's browser. The target application employs regular expressions to make sure no script is being passed through the application to the web page; such a regular expression could be ((?i)script), and the application would replace all matches by this regex by the empty string. An attacker will then create a special payload to bypass this filter: <scriscriptpt>alert(1)</scscriptript> when the applications gets this input string, it will replace all script (case insensitive) by the empty string and the resulting input will be the desired vector by the attacker. In this example, we assume that the application needs to write a particular string in a client-side JavaScript context (e.g., <script>HERE</script>). For the attacker to execute the same payload as in the previous example, he would need to send alert(1) if there was no filtering. The application makes use of the following regular expression as filter ((w+)s*(.*)|alert|eval|function|document) and replaces all matches by the empty string. For example each occurrence of alert(), eval(), foo() or even the string alert would be stripped. An attacker will then create a special payload to bypass this filter: this['al' + 'ert'](1) when the applications gets this input string, it won't replace anything and this piece of JavaScript has exactly the same runtime meaning as alert(1). The attacker could also have used non-alphanumeric XSS vectors to bypass the filter; for example, ($=[$=[]][(__=!$+$)[_=-~-~-~$]+({}+$)[_/_]+($$=($_=!''+$)[_/_]+$_[+$])])()[__[_/_]+__[_+~$]+$_[_]+$$](_/_) would be executed by the JavaScript engine like alert(1) is.", "references": "https://capec.mitre.org/data/definitions/199.html, http://cwe.mitre.org/data/definitions/87.html" },

Specifically I think it's having issues with the * characters following a script element.

As a file with just this content run through pandoc has the same error: <p>, <script>HERE</script> (w+)s\*(.\*)|</p>

Running on MacOS with pandoc version: pandoc 2.13 Compiled with pandoc-types 1.22, texmath 0.12.2, skylighting 0.10.5, citeproc 0.3.0.9, ipynb 0.1.0.1

I appreciate this might be an issue with pandoc but perhaps there is something we can do to escape characters in the description of the threat.

I've been looking at the source code and trying to automate the logic extraction from the threat information. I've got a couple of questions:

• How was the initial threats file produced? In other words, for "Flooding" in threats.json, how were the attributes for the "target" and "condition" extracted? Can you please put me in the right direction? Is this a personal interpretation of the CAPEC information?
• So, for example, the "Flooding" attack is a parent of "TCP Flood," "UDP Flood," and so on. I wonder if one can use the same logic and prefill the "target" and, to some extent fill the "condition" field? I feel they pretty much inherit the same logic/condition, but of course, someone's expert opinion is really appreciated.
• There are some attacks whose fields such as "severity", "likelihood of attack", "example", or "reference" are missing in CAPEC. I wonder if it was you, how would you fill them cautiously and correctly?

Thank you.

I wonder how the acronyms for the threats are made. I can't find a logical relationship between the threat's name and its acronym. Let's take a look at the following example:

DO01 - Flooding

Is there any specific standard for such acronyms?

When using the JSON model format as input to create a report I am getting an error "expecting a list of pytm.Data, item number 0 is a <class 'str'>" (line 213 in the code snippet). https://github.com/izar/pytm/blob/679ea0df19b7b92e7d8359891d53f7ed794d54a3/pytm/pytm.py#L194-L218 My input JSON for data and flows looks like this: "flows": [ { "name": "Actor 1 to Actor 2", "source": "Actor 1", "sink": "Actor 2", "order": 1, "data": [ "Data" ] },{ "name": "Actor 2 to Actor 3", "source": "Actor 2", "sink": "Actor 3", "description": "Another data flow", "data": [ ] } ], "data": [ { "name": "Data", "format": "Text", "isPII": true } ]

Which I believe matches the JSON format when using the JSON output of the CLI tool. Below is the function which leads to calling varData when it creates the Dataflow object. It seems like the varData function doesn't deal with a list of data name strings. input.json in the tests folder doesn't have a data field in it. Since data objects are not in either the boundaries, elements or flows section of the JSON should they be dealt with using their own function e.g. decode_data? https://github.com/izar/pytm/blob/679ea0df19b7b92e7d8359891d53f7ed794d54a3/pytm/json.py#L92-L107

Hi all,

Just noticed that there was no Controls class in the documentation under the docs folder. Not sure if you were aware of it. Just wanted to bring light to it.

After gotten a PR merged, I still cannot see the updated version after pip install pytm. This make we wonder how is versioning done and how packages are released? I get v1.2.1 from pip install. I find 1.2.0 in pyproject.toml and 1.2.0 in setup.py. I can also see a tag called v1.2.0.

I realize that I did not add anything to CHANGELOG.md as part of my changes.