Smart Contract Automated Testing Guidelines
Author: enderlocphan@gmail.com || https://github.com/enderphan94/solidity-pentest/
Last updated
Author: enderlocphan@gmail.com || https://github.com/enderphan94/solidity-pentest/
Last updated
The documents aim to recap my experience in smart contract automated testing besides manual testing. I also put the issues that I faced during the execution, indeed, solutions are given.
For a complex project, you can't just copy paste the single sol file and let it run. To make our life easier, Remix has the localhost connection which allows you to interact with your project in your local machine remotely.
This is something I'm used to doing when the project has a large number of inheritant contracts. Obviously, this makes our life easier than ever by just downloading the git project and do some commands.
Compile your truffle contract if needed with npm install (!remember removing the package-lock.json, if it does have it). Otherwise, the remix wouldn't be able to load all libraries for the contracts that are being called.
Go to WorkSpaces on the left panel and choose "Connect to Localhost"
The message box pops up and you just need to read carefully and copy the command shown in the box to connect your localhost
Important:
path-to-the-shared-folder: needs to be an absolute path
remix-ide-instance-URL: needs to plain with http or https
eg:
Somtimes I still got this error from Remix
Cannot connect to the remixd daemon. Please make sure you have the remixd running in the background.
What I usually do is just switch to a new terminal tab and re-type the remixd command. If needed, you can just uinstall and reinstall the remixd (Close VS-Code to do this, if you have it opened)
https://remix-ide.readthedocs.io/en/latest/remixd.html
The same error but another issue.
https://ethereum.stackexchange.com/questions/78637/cant-connect-remix-ide-using-remixd
Source: https://github.com/crytic/solc-select
You need to just switch the version of solc quickly by a command. The version of solc is kindda painful, depending on the tools and project, you need to use a specific and exact version to compile.. otherwise broke.
During my audit, I've suffered from solc-select installations. I used to install via the shell command, but now they've upgraded to pip3. The thing is that some docker containers do not support pip3, so you would need to install solc-selct into that docker but pip3. Therefore, I've a copied version of the solc-select installed via shell.
Via shell: https://github.com/enderphan94/solc-select-sh-version
Via pip3: https://github.com/crytic/solc-select
Install the version you want
And use it
Check your solc version again
Source: https://github.com/crytic/slither
Detects vulnerable Solidity code with low false positives (see the list of trophies)
Identifies where the error condition occurs in the source code
Easily integrates into continuous integration and Truffle builds
Built-in 'printers' quickly report crucial contract information
Detector API to write custom analyses in Python
Ability to analyze contracts written with Solidity >= 0.4
Intermediate representation (SlithIR) enables simple, high-precision analyses
Correctly parses 99.9% of all public Solidity code
Average execution time of less than 1 second per contract
Slither requires Python 3.6+ and solc, the Solidity compiler.
Using Pip
Using Git
We recommend using an Python virtual environment, as detailed in the Developer Installation Instructions, if you prefer to install Slither via git.
Using Docker
Use the eth-security-toolbox docker image. It includes all of our security tools and every major version of Solidity in a single image. /home/share will be mounted to /share in the container.
To share a directory in the container:
slither <file-name>.sol
Error: Source "@openzeppelin/contracts/utils/Context.sol" not found: File outside of allowed directories.
Fixed: the --allow-path does not work, just download the library and copy them into the dir.. casual way :/
Mythril detects a range of security issues, including integer underflows, owner-overwrite-to-Ether-withdrawal, and others. Note that Mythril is targeted at finding common vulnerabilities, and is not able to discover issues in the business logic of an application. Furthermore, Mythril and symbolic executors are generally unsound, as they are often unable to explore all possible states of a program.
Source: https://github.com/ConsenSys/mythril
Install from Pypi:
Note: In my exprience, I prefer using mythril version installed via pip3 rather than Docker. I've faced so many issues with the docker version, and I decided to switch to pip3 one.
Via pip3: https://github.com/ConsenSys/mythril/blob/develop/README.md#usage
Via Docker: docker run -v $(pwd):/tmp mythril/myth a /tmp/<file-name>.sol --solv 0.5.0
Issue 1
In case the tool gives you this error:
mythril.mythril.mythril_disassembler [ERROR]: The file Token.sol does not contain a compilable contract. mythril.interfaces.cli [ERROR]: input files do not contain any valid contracts
We can use contract address in testnet or ganache https://mythril-classic.readthedocs.io/en/master/security-analysis.html
Ganache: myth a --rpc ganache -a <address>
Issue 2
Evn: MacOS
Just in case the command Pip3 install mythril does not work. I don't remember what happened exactly but something does not work with pip3 in MacOS :)
Use the following command
Issue 3
Error
in self.solidity_files[file_index].full_contract_src_maps IndexError: list index out of range
Just uninstall mythril and reinstall it
This tool takes quite a long time to complete.
Program Exploration: Manticore can execute a program with symbolic inputs and explore all the possible states it can reach
Input Generation: Manticore can automatically produce concrete inputs that result in a given program state
Error Discovery: Manticore can detect crashes and other failure cases in binaries and smart contracts
Instrumentation: Manticore provides fine-grained control of state exploration via event callbacks and instruction hooks
Programmatic Interface: Manticore exposes programmatic access to its analysis engine via a Python API
Note: We recommend installing Manticore in a virtual environment to prevent conflicts with other projects or packages
Option 1: Installing from PyPI:
Option 2: Installing from PyPI, with extra dependencies needed to execute native binaries:
Option 3: Installing a nightly development build:
Option 4: Installing from the master branch:
Option 5: Install via Docker:
Once installed, the manticore CLI tool and Python API will be available.
For a development installation, see our wiki.
Sigle contract in a file
Mutiple contracts in a file
Note:
Manticore takes quite a long time to complete the scan by default, so usually I also use --quick-mode option for quick exploration. Disable gas, generate testcase only for alive states, do not explore constant functions. Disable all detectors.
Source: https://github.com/cleanunicorn/theo
Automatic smart contract scanning which generates a list of possible exploits.
Sending transactions to exploit a smart contract.
Transaction pool monitor.
Web3 console
Frontrunning and backrunning transactions.
Waiting for a list of transactions and sending out others.
Estimating gas for transactions means only successful transactions are sent.
Disabling gas estimation will send transactions with a fixed gas quantity.
Usually I deploy the smart contract in Ganache local network, from that, I can freely have the private keys of many accounts. If you have metamask installed, you can deploy in the testnet and get the private key of the accounts.
Deploy the contract
Run
Enter the private key of the attack's account
Enter the smart contract address
eg:
Souce: https://www.npmjs.com/package/@smartdec/smartcheck
SmartCheck is an extensible static analysis tool for discovering vulnerabilities and other code issues in Ethereum smart contracts written in the Solidity programming language
Copy the contract to a folder
Run
Source: https://github.com/eth-sri/securify2
Supports 38 vulnerabilities (see table below)
Implements novel context-sensitive static analysis written in Datalog
Analyzes contracts written in Solidity >= 0.5.8
To build the container:
To run the container:
contract-dir-full-path: should be the absolute path
eg:
Source: https://github.com/duaraghav8/Ethlint
Ethlint (Formerly Solium) analyzes your Solidity code for style & security issues and fixes them.
In the root directory of your DApp:
This creates .soliumrc.json file, which contains configuration that tells Solium how to lint your project. You should modify this file to configure rules, plugins and sharable configs.
I just usually use this simple setting.
Then you can run
or
Source: https://github.com/streetsidesoftware/cspell
The cspell mono-repo, a spell checker for code.
Source: https://github.com/ConsenSys/surya
Surya is an utility tool for smart contract systems. It provides a number of visual outputs and information about the contracts' structure. Also supports querying the function call graph in multiple ways to aid in the manual inspection of contracts.
Install graphviz
Install surya
Note: I recommend using Surya in VS Code
Here is my list:
Name: vscode-slither
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=samparsky.vscode-slither
Name: Solidity Visual Developer
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=tintinweb.solidity-visual-auditor
Name: Slither
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=trailofbits.slither-vscode
Name: Code Spell Checker
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker
Name: mythril
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=xgwang.mythril
Name: solidity
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=JuanBlanco.solidity
