Add headers and todo

README.md

@@ -11,11 +11,15 @@ widget:
 - text: "pragma solidity ^0.5.7;\n// Context: ParentA | Functions: helloA helloB | Constants: constantA \ncontract HelloWorld is ParentA {"
 ---
 
-# A code autocomplete T5 model for solidity
-- Hello world example to use this model, notice the input `text` includes
+# A code generation T5 model for solidity (web3 smart contract)
+
+## Hello World example
+- A hello world example to use this model, notice the input `text` includes
   - Header solidity version like `pragma solidity ^0.5.7`
   - Ancestor class/library info, e.g. public functions and constants from `ParentA`
   - Contract/Library/Interface declaration header, e.g. `HelloWorld` ended with `{`
+- Or simply use the test widget on the right side of the window and test, however
+  the quality is known to be worse without decoding params
 
 ```python
 # !pip install transformers -q
@@ -38,35 +42,16 @@ print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
 # Expect outcome
 """
 string public constant name = "Hello World";
-string public constant symbol = "HELPER";
-uint8 public constant decimals = 18;
-uint256 public constant initialSupply = 0;
+...
 uint256 public constant override returns (uint256) {
 return initialSupply;
 }
 function initialSupply() public view returns (uint256) {
-return initialSupply;
-}
-function balanceOf(address _owner) public view returns (uint256) {
-return balanceOf(_owner);
-}
-function transfer(address _to, uint256 _value) public returns (bool) {
-balanceOf[msg.sender] -= _value;
-balanceOf[_to] += _value;
-emit Transfer(msg.sender, _to, _value);
-return true;
-}
-function transferFrom(address _from, address _to, uint256 _value) public returns (bool) {
-balanceOf[_from] -= _value;
-balanceOf[_to] += _value;
-emit Transfer(_from, _to, _value);
-return true;
-}
-function approve(address _spender, uint256 _value) public returns (bool)
+...
 """
 ```
 
-
+## Background
 - Base T5 code model: https://huggingface.co/Salesforce/codet5-large
 - Source data: https://huggingface.co/datasets/mwritescode/slither-audited-smart-contracts
   - Processing steps: Clean, contract-level segmentation sepration, split in and out
@@ -116,4 +101,9 @@ function approve(address _spender, uint256 _value) public returns (bool)
     }
     }
     ```
-- Source training code: To be added
+- Source training code: To be added
+
+## Future TODO
+- The model is significantly under-trained because of lack of GPU budget, need 10x colab resources (~$100 for full train)
+- This is quite limited on how the model is used, potentially we could switch to GPT2 decoder-only to compare, but CodeT5 has its strong code optimization
+- Need more classifiers (T5 or BERT alike) to detect potential defects.