Onur SolmazWritings on math and crypto
https://osolmaz.com/
Fri, 15 Feb 2019 16:39:26 +0000Fri, 15 Feb 2019 16:39:26 +0000Jekyll v3.7.4Singular Peak Return<p>In measuring trading performances, a simple benchmark is the <strong>maximum return
that can be obtained by a single trade</strong>. Given the set of times
$T= \{ t_1, \dots, t_n \} \subset\mathbb{R}$, $t_{i-1} \leq t_i$ for all $i=2,\dots,n$
and bijection of prices $P=T\to \mathbb{R}_+$, we have the optimization problem</p>
<script type="math/tex; mode=display">\text{maximize } \frac{P(t_2)}{P(t_1)} \text{ such that } t_1,t_2\in T, t_1\leq t_2</script>
<p>We define</p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{aligned}
T_{\max}:=&\{t_i\mid P(t_{i-1}),P(t_{i+1})\leq P(t_i) ; i = 2,\dots,n-1 \}\\
&\cup\{t_1\mid P(t_2) \leq P(t_1)\}\\
&\cup\{t_n\mid P(t_{n-1}) \leq P(t_n)\}\\
\end{aligned} %]]></script>
<p>as the set of local maxima and the boundaries for which $P$ takes a
greater value than their defined neighborhoods.</p>
<p>The solution to the optimization problem is called <strong>singular peak return</strong> and
defined as</p>
<script type="math/tex; mode=display">\max(\{ P(t_i)/\min(\{P(t_j) \mid j=1,\dots,i\}) \mid t_i \in T_{\max} \})</script>
<p>It can be calculated in $O(n)$ time with this simple algorithm: Start
iterating from the first time, and keep a record of the minimum price
encountered before current time. Whenever a local maximum is encountered, divide
it by the recorded minimum and add it to the result set. After iteration ends,
return the maximum of the result set.</p>
Fri, 15 Feb 2019 00:00:00 +0000
https://osolmaz.com/2019/02/15/singular-peak-return/
https://osolmaz.com/2019/02/15/singular-peak-return/mathGrid Search on Price Smoothing Factors with a Simple Trading Bot<p>In a previous <a href="/2018/12/22/sentiment-classifiers/">blog post</a>, I had introduced
a way of labeling time series data as bullish or bearish. For a time series of
prices $P=\{p_1,\dots,p_n\}$ and corresponding times
$T=\{t_1,\dots,t_n\}$, we introduce for a subset
$\bar{T}=\{t_{X+1},\dots, t_{n-Y}\}$
a mapping $C: \bar{T}\to\{\text{bullish},\text{bearish}\}$ such that</p>
<script type="math/tex; mode=display">% <![CDATA[
C(\bar{T} ; X, Y)
=
\begin{cases}
\text{bullish}&\text{if }
\frac{1}{X} \sum\limits_{j=1}^{X} p_{i-j-1}
\leq
\frac{1}{Y} \sum\limits_{k=1}^{Y} p_{i+k}\\
\text{bearish} & \text{otherwise}
\end{cases} %]]></script>
<p>for $i=X+1,\dots,n-Y$, where $X,Y\in \mathbb{Z}_{>0}$ are the preceding and
following smoothing factors, respectively.</p>
<p>Denoting by $I_i$ the set of all information recorded before
$t_i$, let $\mathcal{I}=\{I_1, \dots, I_n\}$ be the set of classifier inputs.
A trained classifier
$C’: \mathcal{I} \to \{\text{bullish},\text{bearish}\}$
maps every set in the input set to a market sentiment.</p>
<p>The goal of every algorithm proposed for this classification task
is to minimize $|\varepsilon|$ where
$\varepsilon= \{t \mid t\in T; X,Y \in \mathbb{Z}_{>0}; C \neq C’\}$.</p>
<p>The simple trading bot $B$ on the other hand is defined as a finite state machine
executing the orders</p>
<script type="math/tex; mode=display">% <![CDATA[
B(T, C') =
\begin{cases}
\text{buy} & \text{if } c'_{i-1} = \text{bearish} \text{ and } c'_i = \text{bullish} \\
\text{sell} & \text{if } c'_{i-1} = \text{bullish} \text{ and } c'_i = \text{bearish} \\
\text{wait} & \text{otherwise}
\end{cases} %]]></script>
<p>for predicted classes $c’_i$ at $t_i$.
How the buying and selling operations affect the bot’s balances is described
in more detail in the previous <a href="/2018/12/22/sentiment-classifiers/">blog post</a>.</p>
<h2 id="a-grid-search-on-x-and-y-with-linear-svms">A Grid Search on $X$ and $Y$ with Linear SVMs</h2>
<p>For training, we used 3.5 years of hourly Bitcoin prices given below.</p>
<figure>
<p><a href="/img/sentiment_classifiers/btcusd_hourly.svg" target="blank_"><img src="/img/sentiment_classifiers/btcusd_hourly.svg" /></a></p>
</figure>
<p><strong>Model:</strong> Linear Support Vector Machine.</p>
<p><strong>Features:</strong> Given $X$ and $Y$, for every point in the price time series, take
preceding $\max(X, 5)$ points and
<a href="https://en.wikipedia.org/wiki/Feature_scaling#Standardization">standardize</a> them.</p>
<p><strong>Targets:</strong> Classes calculated by $C$.</p>
<p>The dataset is split 75-25% (not shuffled) for training/testing. Below are the accuracies
obtained for each $(X,Y) \in [1,100]\times [1,100]$.</p>
<figure>
<p><a href="/img/grid_search_smoothing_factors/fig1.svg" target="blank_"><img src="/img/grid_search_smoothing_factors/fig1.svg" /></a>
Accuracies obtained through the grid search. Maximum accuracy was given by
$X=100$ and $Y=1$ at 93.6%.</p>
</figure>
<h2 id="monte-carlo-simulation-with-resulting-accuracies">Monte-Carlo Simulation with Resulting Accuracies</h2>
<p>Before looking into how the trading bot performs with the trained classifier,
let’s first simulate a trading bot that gets the sentiment right $accuracy\%$
of the time. We do this by implementing a Poisson process and running a Monte
Carlo simulation to get the expected value of accuracies, as described
in the previous <a href="/2018/12/22/sentiment-classifiers/">blog post</a>.</p>
<figure>
<p><a href="/img/grid_search_smoothing_factors/fig2.svg" target="blank_"><img src="/img/grid_search_smoothing_factors/fig2.svg" /></a>
Monte Carlo simulation of the returns (ran 20 times and averaged) if the
classifier went wrong according to a Poisson distribution.</p>
</figure>
<p>The returns range between over 1 billion times to actual loss below 1x. The
results look optimistic: more than half of the values are greater than 1000x. Is
it too good to be true?</p>
<h2 id="actual-returns-with-the-trained-classifiers">Actual Returns with the Trained Classifiers</h2>
<p>Kind of… The maximum return we get is 152x:</p>
<figure>
<p><a href="/img/grid_search_smoothing_factors/fig3.svg" target="blank_"><img src="/img/grid_search_smoothing_factors/fig3.svg" /></a>
Actual returns obtained when bought and sold according to the input from trained
classifiers.</p>
</figure>
<p>There seems to be a region $25<X<45$ where trading yields 80x return on average.
The highest returns are obtained for $1<Y<25$ and $80<Y<100$, but there is not
enough evidence to draw conclusions of a pattern. To do that, the bot should be
trained and run on more price data.</p>
<p>The reason for the drastic difference from the simulation is that classifiers
don’t err through a Poisson process: there is a dependence between consecutive
events. Errors are clumped together, meaning that an error happening increases
the likelihood of it happening in the next hour.</p>
<p>Another interesting result was that trading with the actual classifier yielded
much higher returns than the simulation for certain values, e.g. towards
top-left.</p>
<p>Being optimistic, we could make the assumption that the
cryptocurrency market would perform similarly in the next 3-4 years. Deploying
the bot for the apparently optimal range, ~80x in 3.5 years equals an average
return of ~3.5x/year.</p>
<p>Or we could just buy and hold Bitcoin, because minimum and maximum prices for the data
above were \$200 versus \$19847, and selling at the right time would have
nevertheless yielded ~100x returns. The problem is, of course, knowing when to buy
and sell. The value of such a bot arguably comes from the fact that it saves an
investor from the hassle and opportunity cost of keeping track of the market.</p>
<p>A proposed heuristic:
Choose a <em>volatile</em> and <em>promising</em> asset with more than 10,000 hours of history.
Train over past data, run, and forget about it for a couple of years. Hopefully,
it will have generated enough returns to compensate running costs.</p>
<h2 id="summary">Summary</h2>
<p>The classifier and trading bot given in the previous blog post were formalized.
A grid search was performed over smoothing factors with linear SVMs. The returns
for resulting accuracies were simulated with the assumption of a Poisson
distribution. The actual returns were given and compared with the simulation.</p>
Mon, 11 Feb 2019 00:00:00 +0000
https://osolmaz.com/2019/02/11/grid-search-smoothing-factors/
https://osolmaz.com/2019/02/11/grid-search-smoothing-factors/mathcryptoInflation Curvature in Bitcoin Supply<p>New bitcoins are minted with every new block in the Bitcoin blockchain, called “block
rewards”, in order to incentivize mining by making it more profitable. This
inflates Bitcoin’s supply in a predictable manner.
Imagine that the inflation rate were constant and the transactions between
miners and users stopped at some point:
miners’ share of the total Bitcoin supply would approach 100% as the number
of blocks goes to infinity.</p>
<p>But that’s not the case, and said boost is valid only in the short term. In
fact, inflation rate decreases geometrically and there will come a time when
it’s effectively negligible. Whereas block rewards constituted the majority of
what a miner receives from a block, these will in time converge to zero and only
transaction fees will remain. The effect will be increased transaction fees, assuming
the mining market is already saturated enough.</p>
<p>There have been some confusion of the terminology, like people calling Bitcoin
deflationary. Bitcoin is in fact not deflationary—that implies negative
inflation rate. Bitcoin rather has negative <strong>inflation curvature</strong>: Bitcoin’s
inflation rate decreases monotonically.</p>
<p>An analogy from elementary physics should clear things up: Speaking strictly in
terms of monetary inflation,</p>
<ul>
<li><em>displacement</em> is analogous to <em>inflation/deflation</em>, as in total money
minted/burned, without considering a time period. Dimensions: $[M]$.</li>
<li><em>Velocity</em> is analogous to <em>inflation rate</em>, which defines total money minted/burned
in a given period. Dimensions: $[M/T]$.</li>
<li><em>Acceleration</em> is analogous to <em>inflation curvature</em>, which defines the total
change in inflation rate in a given period. Dimensions: $[M/T^2]$.</li>
</ul>
<p>Given a supply function $S$ as a function of time, block height, or any variable
signifying progress,</p>
<ul>
<li>inflation is a positive change in supply, $\Delta S$, and deflation, $-\Delta S$.</li>
<li>Inflation rate is the first derivative of supply, $S’$.</li>
<li>Inflation curvature is the second derivative of supply, $S’’$.</li>
</ul>
<p>In Bitcoin, we have the supply as a function of block height:
$S:\mathbb{Z}_{\geq 0} \to \mathbb{R}_+$.
But the function itself is defined by the arithmetic<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup> initial value problem</p>
<script type="math/tex; mode=display">S'(h) = \alpha^{\lfloor h/\beta\rfloor} R_0
,\quad
S(0) = 0
\tag{1}</script>
<p>where $R_0$ is the initial inflation rate, $\alpha$ is the rate by which the
inflation rate will decrease, $\beta$ is the milestone number of blocks at
which the decrease will take place, and $\lfloor \cdot \rfloor$ is the floor
function. In Bitcoin, we have $R_0 = 50\text{ BTC}$,
$\alpha=1/2$ and $\beta=210,000\text{ blocks}$. Here is what it looks like:</p>
<figure>
<p><a href="/img/inflation_curvature_bitcoin_supply/fig2.svg" target="blank_"><img src="/img/inflation_curvature_bitcoin_supply/fig2.svg" /></a>
Bitcoin inflation rate versus block height.</p>
</figure>
<p>We can directly compute inflation curvature:</p>
<script type="math/tex; mode=display">% <![CDATA[
S''(h) =
\begin{cases}
\frac{\ln(\alpha)}{\beta} \alpha^{h/\beta} & \text{if}\quad
h\ \mathrm{mod}\ \beta = 0 \quad\text{and}\quad h > 0\\
0 & \text{otherwise}.
\end{cases} %]]></script>
<p>$S’’$ is nonzero only when $h$ is a multiple of $\beta$. For $0 < \alpha < 1$,
$S’’$ is either zero or negative, which is the case for Bitcoin.</p>
<p>Finally, we can come up with a closed-form $S$ by solving the initial value
problem (1):</p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{aligned}
S(h) &= \sum_{i=0}^{\lfloor h/\beta\rfloor -1} \alpha^{i} \beta R_0
+ \alpha^{\lfloor h/\beta\rfloor} (h\ \mathrm{mod}\ \beta) R_0 \\
&= R_0 \left(\beta\frac{1-\alpha^{\lfloor h/\beta\rfloor}}{1-\alpha}
+\alpha^{\lfloor h/\beta\rfloor} (h\ \mathrm{mod}\ \beta) \right)
\end{aligned} %]]></script>
<p>Here is what the supply function looks like for Bitcoin:</p>
<figure>
<p><a href="/img/inflation_curvature_bitcoin_supply/fig1.svg" target="blank_"><img src="/img/inflation_curvature_bitcoin_supply/fig1.svg" /></a>
Bitcoin supply versus block height.</p>
</figure>
<p>And the maximum number of Bitcoins to ever exist are calculated by taking the
limit</p>
<script type="math/tex; mode=display">\lim_{h\to\infty} S(h)
= \sum_{i=0}^{\infty} \alpha^{i} \beta R_0
= \frac{\beta R_0}{1-\alpha}
= 21,000,000\text{ BTC}.</script>
<h1 id="summary">Summary</h1>
<p>The concept of inflation curvature was introduced. The confusion regarding
Bitcoin’s inflation mechanism was cleared with an analogy. The IVP defining
Bitcoin’s supply was introduced and solved to get a closed-form
expression. Inflation curvature for Bitcoin was derived.
The maximum number of Bitcoins to ever exist was derived and computed.</p>
<div class="footnotes">
<ol>
<li id="fn:1">
<p>Because $S$ is defined over positive integers. <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>
Sat, 09 Feb 2019 00:00:00 +0000
https://osolmaz.com/2019/02/09/inflation-curvature-bitcoin-supply/
https://osolmaz.com/2019/02/09/inflation-curvature-bitcoin-supply/mathcryptoeconomicsPyramid or Stump? Bonding Curve Parameters Matter<p>A bonding curve is a mechanism that enables bootstrapping a new currency based
on another currency, called the base currency.
It’s like a fixed-exchange rate system in terms of control over the currency;
but instead of having a fixed-exchange rate, an authority increases the exchange
rate with increasing supply of the minted currency. The rate of increase is
determined by the bonding curve which strictly defines the relationship between
the exchange rate and the supply. Conversely, the exchange rate drops when the
supply decreases.</p>
<p>The total amount of base currency the authority keeps at any time is called
“reserve”.
Since the mechanism described mints the new currency, we use the term “price”
instead of exchange rate, as in minting price.</p>
<p>When designing such a system, it’s not just the type of the bonding curve that
matters, but also the parameters it uses. These parameters determine the
distribution of wealth among early and late buyers of the bonding curve.
In other words, a linear bonding curve’s initial price and slope determine
whether the distribution is a <em>pyramid</em> or a <em>stump</em>.</p>
<figure>
<p><a href="/img/pyramid_or_stump/fig1.svg" target="blank_"><img src="/img/pyramid_or_stump/fig1.svg" /></a></p>
</figure>
<p>Pyramid means a distribution
where early buyers can mint for much cheaper and have a lot more of the new
currency than late buyers, whereas stump means a comparatively more even
distribution, although early buyers still enjoy some advantage.
The steeper the bonding curve, the more pyramid-like the distribution.</p>
<h3 id="preliminaries">Preliminaries</h3>
<p>In a linear bonding curve, the price is a linear function of the supply:</p>
<script type="math/tex; mode=display">P(S) = P_0 + \Delta_p S</script>
<p>where $P$ is price, $S$ is supply, $P_0$ is the initial price and $\Delta_p$ is the price increment.</p>
<p>Were the price constant, we would have had the following relationship
between the amount of base currency $B$, minted currency $M$ and price $P$</p>
<script type="math/tex; mode=display">P = \frac{B}{M}</script>
<p>But since we have a changing price, we have the following differential equation instead:</p>
<script type="math/tex; mode=display">P = \frac{d B}{d M}</script>
<p>That’s why the area under the bonding curve equals total reserve for a given price.
Thus we calculate total reserve $R$ as a function of the total supply as</p>
<script type="math/tex; mode=display">R(S) = \int_0^S P(\tilde{S}) \, d\tilde{S} = P_0 S + \frac{1}{2} \Delta_p S^2</script>
<p>Inverting the function, we get</p>
<script type="math/tex; mode=display">S(R) = \frac{\sqrt{P_0^2+2 \Delta_p R}-P_0}{\Delta_p}</script>
<p>Plugging this into the price-supply function, we can get price as a function
of reserve</p>
<script type="math/tex; mode=display">P(R) = \sqrt{P_0^2+2\Delta_p R}</script>
<p>While choosing parameters, expressing price in terms of the underlying reserve
is more useful because we have a better feeling of the base currency compared to
the newly minted one. If we can come up with projections of market attributes, we
can reason better about how steep the price should
increase. Below are three metrics that can aid in this decision.</p>
<h3 id="metric-1-price-increase-after-a-certain-reserve-amount">Metric 1: Price increase after a certain reserve amount</h3>
<p>The first metric that comes to mind is the increase in price after the
investment of a certain amount $B$ of the base currency. The founder of the
bonding curve may design it according to a principle, e.g.</p>
<blockquote>
<p>The price should increase 2x with R reserve.</p>
</blockquote>
<p>Below is a graph that demonstrates this:</p>
<figure>
<p><a href="/img/pyramid_or_stump/fig3.svg" target="blank_"><img src="/img/pyramid_or_stump/fig3.svg" /></a>
Price versus reserve for different price increments, plotted using the formula above.</p>
</figure>
<p>For example, if the founder wants a 2x price increase for 3000 base currency
invested in the bonding curve, then
they should choose $\Delta_p \approx 0.0005$.</p>
<h3 id="metric-2-initial-slope">Metric 2: Initial slope</h3>
<p>The initial slope of the price-reserve curve is good for comparing
different sets of parameters. It’s equal to</p>
<script type="math/tex; mode=display">\frac{dP}{dR}\bigg\rvert_{R=0} = \frac{\Delta_p}{P_0}</script>
<p>which is a straightforward metric that uses both parameters.</p>
<h3 id="metric-3-weak-hand-return">Metric 3: Weak hand return</h3>
<p>Another metric is the return obtained when</p>
<ol>
<li>the first ever buyer mints $M_0$ with $B$ base currency,</li>
<li>another buyer mints $M_1 < M_0$ (since price has increased) with $B$ base currency,</li>
<li>and the first buyer burns $M_0$ for profit.</li>
</ol>
<p>The return is calculated as</p>
<script type="math/tex; mode=display">return = \frac{\int_{M_1-M_0}^{M_1} P(S)\,dS }{\underbrace{\int_0^{M_0} P(S)\,dS}_{=B}}</script>
<p>where $M_0 = S(B)$ and $M_1 = S(2B)$. It eventually yields</p>
<script type="math/tex; mode=display">return = \frac{1}{\Delta_p B} \left(\sqrt{P_0^4 + 6P_0^2\Delta_p B + 8\Delta_p^2B^2}
+ P_0\left(\sqrt{P_0^2+2\Delta_p B}-\sqrt{P_0^2+4\Delta_p B}\right)
- P_0^2
\right)-1</script>
<p>One interesting result is</p>
<script type="math/tex; mode=display">\lim_{B\to 0} return = 1
\quad\text{and}\quad
\lim_{B\to\infty} return = 2\sqrt{2} - 1 \approx 1.8284\dots</script>
<p>For all positive values of $B$, the return is between 1 and 1.8284.
$B$ is supposed to be a purchase made by an
average buyer. In other words, it’s the expected value of an investment to the
bonding curve.</p>
<p>For a given B, the closer the return is to 1.8284, the more pyramid-like the
distribution. The closer it is to 1, the more stump-like the distribution.</p>
<figure>
<p><a href="/img/pyramid_or_stump/fig2.svg" target="blank_"><img src="/img/pyramid_or_stump/fig2.svg" /></a>
Weak hand return versus B. Price increment’s relationship to the
initial price is a determining factor, and returns have been plotted for
different price increments. For $B=5000$, we can say that around $\Delta_p=0.01$ is
roughly where the distribution stops looking less like a pyramid and more like a stump.</p>
</figure>
<h3 id="example">Example</h3>
<p>As of the beginning of 2019, the most sizeable deployment of a bonding curve ever was
the
<a href="https://etherscan.io/address/0xb3775fb83f7d12a36e0475abdd1fca35c091efbe">PoWH3D contract</a>
on Ethereum. Here is my
<a href="https://osolmaz.com/2018/08/03/token-bonding-1/">previous analysis</a> of it.
At it’s height on July 27, 2018, the contract had more than 82,000 ETH, then
equal to ~$35 million of value.</p>
<p>Is PoWH3D a pyramid or a stump? The parameters are $P_0$ = 1e-7 ETH and
$\Delta_p$ = 1e-8 ETH. Let’s plug them in and see.</p>
<figure>
<p><a href="/img/pyramid_or_stump/fig4.svg" target="blank_"><img src="/img/pyramid_or_stump/fig4.svg" /></a>
Price versus reserve for PoWH3D, plotted up to reserve all-time high.</p>
</figure>
<p>It seems like the price starts at 0, but in reality, it has
increased so much that one can’t tell the difference. After 1 ETH in the
contract, the price increases ~1400x; after 10 ETH, ~4400x; after 100 ETH, ~14,000x;
after 1000 ETH, ~44,000x; after 10,000 ETH, ~140,000x and so on.
The price triples with each increase in the order of magnitude.
Someone who invested when there was only ~100 ETH in the contract could have
sold it for 28x profit at its height, not taking dividends into account.</p>
<p>Initial slope of the price-reserve curve is equal to 0.1, for the record.</p>
<p>What about the weak hand return? Below is a graph that shows for up to 1 ETH:</p>
<figure>
<p><a href="/img/pyramid_or_stump/fig5.svg" target="blank_"><img src="/img/pyramid_or_stump/fig5.svg" /></a>
Weak hand return versus investment size for PoWH3D. $B$ is in ETH.</p>
</figure>
<p>As you can see, it shoots up to the maximum value really quickly.</p>
<p>Interpreting these results, we can arrive at the conclusion that PoWH3D is a
pyramid rather than a stump, and the price increase is very steep.</p>
<h3 id="summary">Summary</h3>
<p>The price-reserve relationship, derived from the price-supply relationship,
is more useful in determining linear bonding
curve parameters. More informed decisions can be made based on this curve, using
metrics such as initial slope and price increase at a certain amount of
reserve.
Another metric has been introduced, which calculates the returns after
selling when an investment of size equal to one’s own is made to the bonding
curve. A real-life example has been analyzed.</p>
Sun, 27 Jan 2019 00:00:00 +0000
https://osolmaz.com/2019/01/27/pyramid-or-stump/
https://osolmaz.com/2019/01/27/pyramid-or-stump/matheconomicscryptoBenchmarking Market Sentiment Classifiers<p>A very simple trading bot is one that buys when a market turns from bearish to
bullish and sells when it turns from bullish to bearish. The entity that informs
the bot about this sentiment is called a <em>sentiment classifier</em>.</p>
<p>Given your trading frequency, there is in fact a theoretical upper limit to how
much you can earn by trading, called the <em>peak return</em>. It can act as a
reference to benchmark different types of classifiers.</p>
<h2 id="preliminaries">Preliminaries</h2>
<p><strong>Definition:</strong> The <strong>trading frequency</strong> $f$ is the <em>maximum</em> frequency at which a trader
chooses to trade. For example, if a trader limits buy/sell operations for a
single asset to once a day, than the trading frequency is $1/\text{day}$. The
trader can choose not to trade for certain days—this doesn’t change the
trading frequency.</p>
<p><strong>Definition:</strong> The <strong>price</strong> of an asset $p$ is a piecewise linear function
$p:\mathbb{R}\to\mathbb{R}$ where $p$ is linear in constant intervals $1/f$
and $p(t)>0$ for all $t\in\mathbb{R}$,</p>
<p><strong>Definition:</strong> Given initial account balance $C_0$ and asset balance $A_0$,
a <em>purchase</em> of $\bar{A}$ assets with $\bar{C}$ capital at time $t_0$ renders
the balances</p>
<script type="math/tex; mode=display">C_1 \leftarrow C_0 - p(t_0)\bar{A} \quad \text{and}\quad A_1 \leftarrow A_0 + p(t_0)\bar{A}.</script>
<p>where the relation $\bar{C} = p(t_0) \bar{A}$ holds. Similarly, a sale of $\bar{A}$ assets
for $\bar{C}$ capital at time $t_0$ renders the balances</p>
<script type="math/tex; mode=display">C_1 \leftarrow C_0 + p(t_0)\bar{A} \quad \text{and}\quad A_1 \leftarrow A_0 - p(t_0)\bar{A}</script>
<p>Finally,
a <em>full purchase</em> at time $t_0$ renders the balances</p>
<script type="math/tex; mode=display">C_1 \leftarrow 0 \quad \text{and}\quad A_1 \leftarrow A_0 + C_0 / p(t_0),</script>
<p>and a <em>full sale</em> renders the balances</p>
<script type="math/tex; mode=display">C_1 \leftarrow C_0 + p(t_0)A_0 \quad\text{and}\quad A_1 \leftarrow 0.</script>
<p>A purchase $B$ taking place at time $t_n$, spending $\bar{C}_n$ amount of capital is
denoted by the 3-tuple $(t_n, p(t_n), \bar{C}_n)$.</p>
<p>Similarly, a sale $S$ taking place at time $t_n$, selling $\bar{A}_n$ amount of assets is
denoted by the 3-tuple $(t_n, p(t_n), \bar{A}_n)$.</p>
<p><strong>Definition:</strong> A <strong>trade</strong> refers to either a purchase or a sale. A
<strong>trade cycle</strong> $C$ is a 2-tuple of a consecutive full purchase and full sale
$(B_n, S_{n+1})$.
<em>Trade cycle</em> will be referred to as <em>cycle</em> for brevity.</p>
<p><strong>Definition:</strong> The <strong>return</strong> $R$ of a cycle taking place between $t_n$ and
$t_{n+1}$ is calculated as $p(t_{n+1})/p(t_n)$. In other words, the return of a
cycle is how much it multiplies an initial capital, generally denoted by a
number followed by x, e.g. 3x.
Given a series of cycles
$\{C_1, C_2, \dots, C_n \}$ with returns
$\{R_1, R_2, \dots, R_n \}$ their combined return is equal to
$R_1 R_2 \cdots R_n$.</p>
<h2 id="fundamental-theorem-of-trading-and-peak-return">Fundamental Theorem of Trading and Peak Return</h2>
<p>Most traders are familiar with the saying <em>“buy the dip, sell the rally”</em>, which
is very reasonable advice if you can guess when the dips and rallies are. We
nevertheless know exactly when the <strong>past</strong> dips and rallies are, and can use them
to calculate the peak return for a given period in the past.</p>
<p><strong>Fundamental Theorem of Trading (FTT):</strong> For a given price function $p$ and time
period $[a, b]$
the return is maximized when assets are fully bought at local minima (dips),
and fully sold at local maxima (rallies). This value is called the <strong>peak
return</strong> $PR$.</p>
<p>We assume that trading starts with a full purchase and ends with a full sale. If</p>
<script type="math/tex; mode=display">\lim_{\epsilon\to 0} p(a+\epsilon)- p(a) > 0</script>
<p>the first purchase takes place at $a$; otherwise it takes place at
the next local
minimum. If the last extremum before $b$ is a local minimum, the last
sale takes place at $b$.</p>
<p>For a given set of points, it is easy to calculate peak return algorithmically.
It is also important to
note the more general analytic definition that extends beyond the
piecewise-linearity assumption:</p>
<script type="math/tex; mode=display">PR[p; a,b] = 1 + \frac{1}{p(a)}\int_a^b \rho(t) H(p'(t)) p'(t) \, dt</script>
<p>where $H$ is the Heaviside step function, $p’$ is $p$’s first derivative, and
$\rho(t)$ is defined as the multiplication of returns of cycles taking place before
$t$.</p>
<figure>
<p><a href="/img/sentiment_classifiers/peak_return_example.svg" target="blank_"><img src="/img/sentiment_classifiers/peak_return_example.svg" /></a>
Prices given by the function $p(x)=4\sin(\pi x) + 2x + 8$ (left), and
corresponding peak returns in time (right).</p>
</figure>
<p>To summarize, peak return is the maximum return possible for a given price function.</p>
<h2 id="sentiment-classifiers">Sentiment Classifiers</h2>
<p>It is not possible to predict local minima and maxima of a price
function precisely. However, it is possible to predict them to a degree of
confidence.</p>
<p>Since we can’t know with certainty whether the current price value is a local
minimum or maximum, we name them differently; namely <strong>buy signal</strong> instead of
a possible local minimum and <strong>sell signal</strong> instead of a possible local
maximum.</p>
<p>We conceptualize a <strong>sentiment classifier</strong> that takes some input, and predicts the
market sentiment as bullish (the price will be higher at some point in the
future) or bearish (the price will be lower at some point in the future).</p>
<p><strong>Definition:</strong> A <strong>perfect sentiment classifier</strong> (PSC) is a hypothetical classifier
that predicts with 100% accuracy whether</p>
<script type="math/tex; mode=display">\lim\limits_{\epsilon \to 0} p(\bar{t} +\epsilon)
\stackrel{?}{\gtrless}
p(\bar{t})</script>
<p>for a given time $\bar{t}$.</p>
<h2 id="a-simple-trading-bot">A Simple Trading Bot</h2>
<p>We conceptualize a trading bot using a sentiment classifier, which starts
with some initial capital, fully buys when the sentiment turns from bearish to
bullish and fully sells when the sentiment turns from bullish to bearish.</p>
<p><strong>Lemma:</strong> A trading bot can achieve peak return with a PSC.</p>
<h2 id="smoothing-sentiment-classifiers">Smoothing Sentiment Classifiers</h2>
<p>The more a sentiment classifier diverges from a perfect one, the lower are the
returns of the trading bot that uses it.</p>
<p><strong>Definition:</strong> A <strong>smoothing sentiment
classifier</strong> (SSC) is one that can accurately predict whether</p>
<script type="math/tex; mode=display">\frac{1}{\beta}\int_{\bar{t}}^{\bar{t}+\beta} p(t)\, dt
\stackrel{?}{\gtrless}
\frac{1}{\alpha}\int_{\bar{t}-\alpha}^{\bar{t}} p(t)\, dt</script>
<p>for a given time $\bar{t}$, a period of length $\alpha$ preceding $\bar{t}$ and a
period of length $\beta$ following $\bar{t}$.
In other words, the smoothing classifier predicts the relationship between
means of the price function preceding and following current time.</p>
<p>For a piecewise constant price function consisting of a discrete set of
values $\boldsymbol{p} = [p_0, p_1, \dots, p_n]$ and corresponding equidistant
time values $\boldsymbol{t} = [t_0, t_1, \dots, t_n]$, an SSC can simply be
rephrased as one that predicts whether</p>
<script type="math/tex; mode=display">\frac{1}{Y} \sum\limits_{k=1}^{Y} p_{i+k}
\stackrel{?}{\gtrless}
\frac{1}{X} \sum\limits_{j=1}^{X} p_{i-j-1}</script>
<p>for time $t_i$, $X$ preceding points including $t_i$ and $Y$ following points
excluding $t_i$. This classification is visualized in the following figure:</p>
<figure>
<p><a href="/img/sentiment_classifiers/labeling_ex.svg" target="blank_"><img src="/img/sentiment_classifiers/labeling_ex.svg" /></a>
A visual explanation for the relationship given above. For the selected point, the average of
the following Y points is greater than the average of the preceding X points,
making the sentiment bullish.</p>
</figure>
<p>To better visualize the
concept, we use 3.5 years of hourly historical Bitcoin price data from Coinbase:</p>
<figure>
<p><a href="/img/sentiment_classifiers/btcusd_hourly.svg" target="blank_"><img src="/img/sentiment_classifiers/btcusd_hourly.svg" /></a>
The peak return for this price graph is ~2.02e33x.</p>
</figure>
<p>We plot the classes for different values of $X$ and $Y$ to observe the
smoothing effect:</p>
<figure>
<p><a href="/img/sentiment_classifiers/sentiment_example.svg" target="blank_"><img src="/img/sentiment_classifiers/sentiment_example.svg" /></a>
Sentiment classes for $X=Y=1$ (top) and $X=Y=10$ (bottom).</p>
</figure>
<p>Notice that smoothed sentiments make more sense visually in terms of
reflecting long term trends.</p>
<h3 id="the-effect-of-smoothing-on-the-number-of-trade-cycles">The Effect of Smoothing on the Number of Trade Cycles</h3>
<p>Smoothing increases the overall length of contiguous bullish or bearish periods,
and decreases the number of signals for a given price function. Therefore, the
number of trade cycles also decreases. Below is a graph of the number of cycles
for increasing $X$ and constant $Y$:</p>
<figure>
<p><a href="/img/sentiment_classifiers/n_cycles_x.svg" target="blank_"><img src="/img/sentiment_classifiers/n_cycles_x.svg" /></a>
Number of trade cycles divided by the maximum number of trade cycles (given by
$X=Y=1$) for the same hourly BTC-USD pair. For $Y=1$ and increasing $X$.</p>
</figure>
<p>It can be observed from the graph that the relationship between smoothing
factors and number of cycles is roughly hyperbolic, given by the following
function</p>
<script type="math/tex; mode=display">f(X) = \left[\frac{1+b}{X+b}\right]^a</script>
<p>which ensures that $f(1)=1$. Here, $Y$ is kept constant at 1 and $X$ is allowed
to increase.</p>
<p>One can do the same vice versa for $Y$ by keeping $X$ constant, and find the parameters</p>
<script type="math/tex; mode=display">g(Y) = \left[\frac{1+d}{Y+d}\right]^c</script>
<p>Then, the multi-variable function for the number of cycles can be approximated
by</p>
<script type="math/tex; mode=display">h(X, Y) = \left[\frac{1+b}{X+b}\right]^a \left[\frac{1+d}{Y+d}\right]^c</script>
<p>which can be used later in closed-form statistical derivations. Below is a plot
of the relative error for the BTC-USD pair:</p>
<figure>
<p><a href="/img/sentiment_classifiers/n_cycles_error.svg" target="blank_"><img src="/img/sentiment_classifiers/n_cycles_error.svg" width="70%" /></a>
Relative error of the approximation given by the function $h$ above.
The parameters are
a=0.59219397, b=0.5894406, c=0.5629668, d=0.58416813.</p>
</figure>
<h3 id="the-effect-of-smoothing-on-final-return">The Effect of Smoothing on Final Return</h3>
<p>We can observe the effect of smoothing on the final return of a
trading bot, when we
plot the returns for varying values of $X$ and $Y$.</p>
<figure>
<p><a href="/img/sentiment_classifiers/btcusd_hourly_XY.svg" target="blank_"><img src="/img/sentiment_classifiers/btcusd_hourly_XY.svg" /></a>
Final returns for the price graph above, and varying $X$ and $Y$. Returns are on
a logarithmic scale. Lower-left corner of the contour plot corresponds to peak
return, which is diverged from with increasing $X$ and $Y$.</p>
</figure>
<p>Final return decreases super-exponentially with
increasing smoothing factor.</p>
<h2 id="the-effect-of-fuzzing-classifier-output">The Effect of Fuzzing Classifier Output</h2>
<p>In the previous sections, we introduced a hypothetical classifier that work with
100% accuracy. However, a classifier having access to a finite amount of
information can never achieve that kind of success rate. To see how decreased
accuracy affects final returns, we can introduce errors retroactively to a PSC
output. In other words, we “fuzz” the 100% correct case by introducing random errors.
The process can be summarized as:</p>
<ol>
<li>Obtain correct sentiment classes.</li>
<li>Iterate through array elements and
change the value if a randomly generated number is smaller than
Z%, that is, the fuzzing amount. This is analogous to a Poisson process.</li>
<li>Calculate final return with the fuzzed classes.</li>
<li>Repeat steps 1-3 many times to obtain a large enough distribution of final
returns for Z, and take the average.</li>
<li>Repeat steps 1-4 for different Z.</li>
</ol>
<p>After running the process for the prices given in the previous section,
we arrive at the following result:</p>
<figure>
<p><a href="/img/sentiment_classifiers/fuzzing_example.svg" target="blank_"><img src="/img/sentiment_classifiers/fuzzing_example.svg" /></a>
Final returns are on a logarithmic scale, and are also calculated for different
smoothing factors. The black line corresponds to 1x (or 0%) final return.</p>
</figure>
<p>On average, final return decreases exponentially with increasing fuzzing amount,
given that the errors have a Poisson distribution. The rate of decay
decreases with increasing smoothing factor. The lines intersect at 50% fuzzing
with ~7.5x return. 0% return is obtained by fuzzing
around 50~60% for different smoothing factors. These values are not
universal and depend on the given prices.</p>
<p>This and the previous relations can be utilized in
hyperparameter optimization when building sentiment classifiers with machine
learning models.</p>
<h2 id="conclusion">Conclusion</h2>
<p>We introduced fundamental concepts such as peak return and
sentiment classifier for a more mathematical understanding of
trading.
By breaking down the decision-making process of trading into different entities,
we can reason more efficiently when choosing models. By introducing mathematical
limits to trading performance, we can benchmark and compare different models.
Overall, the results obtained should be helpful in developing statistically
sound trading algorithms.</p>
Sat, 22 Dec 2018 00:00:00 +0000
https://osolmaz.com/2018/12/22/sentiment-classifiers/
https://osolmaz.com/2018/12/22/sentiment-classifiers/mathcryptoThe Anatomy of a Block Stuffing Attack<p><em>Block stuffing</em> is a type of attack in blockchains where an attacker submits
transactions that deliberately fill up the block’s gas limit and stall other
transactions. To ensure inclusion of their transactions by miners, the
attacker can choose to pay higher transaction fees. By controlling the
amount of gas spent by their transactions, the attacker can influence the number
of transactions that get to be included in the block.</p>
<p>To control
the amount of gas spent by the transaction, the attacker utilizes a special
contract. There is a function in the contract which takes as input the amount of
gas that the attacker wants to burn. The function runs meaningless
instructions in a loop, and either returns or throws an error when the desired
amount is burned.</p>
<p>For example let’s say that the average gas price has been 5 Gwei in the last
10 blocks. In order to exert influence over the next block, the attacker needs to
submit transactions with gas prices higher than that, say 100 Gwei. The higher
the gas price, the higher the chance of inclusion by miners.
The attacker can choose to divide the task of using 8,000,000 gas—current
gas limit for blocks—into as
many transactions as they want. This could be 80 transactions with
100,000 gas expenditure, or 4 transactions with 2,000,000 gas expenditure.</p>
<p>Deciding on how to divide the task is a matter of maximizing the chance of
inclusion, and depends on the factors outline below.</p>
<h2 id="miners-strategy-for-selecting-transactions">Miners’ strategy for selecting transactions</h2>
<p>Miners want to maximize their
profit by including transactions with highest fees. In the current PoW
implementation of Ethereum, mining the block takes significantly more time
than executing the transactions. So let’s assume
all transactions in the pool are trivially executed as soon as they arrive
and miners know the amount of gas each one uses.</p>
<p>For miners, maximizing profit is an
<a href="https://en.wikipedia.org/wiki/Packing_problems">optimum packing problem</a>.
Miners want to choose a subset of the transaction pool that gives them
maximum profit per block. Since there are at least tens of thousands of
transactions in the pool at any given time, the problem can’t be solved by
brute-forcing every combination. Miners use algorithms that test a
feasible number of combinations and select the one giving the highest reward.</p>
<p>A block stuffer’s main goal is to target the selection process by
crafting a set of transactions that has the highest chance of being picked
up by miners in a way that will deplete blocks’ gas limits.
They can’t devise a 100% guaranteed strategy since each miner
can use a different algorithm, but they can find a sweet spot by testing out the
whole network.</p>
<p>(In a PoS system, our assumptions would be wrong since executing
transactions is not trivial compared to validating blocks. Validators would
need to develop more complex strategies depending on the PoS implementation.)</p>
<h2 id="the-transactions-the-attacker-wants-to-stall">The transactions the attacker wants to stall:</h2>
<p>It could be so that the
attacker wants to stall transactions with a specific contract. If the
function calls to that contract use a distinctively high amount of gas, say
between 300,000 and 500,000, then the attacker has to stuff the block in a
way that targets that range.</p>
<p>For example, the attacker can periodically submit $n$ transactions
$\{T_1, T_2,\dots, T_{n-1}, T_n\}$ with very high prices where</p>
<script type="math/tex; mode=display">\sum\limits_{i=1}^{n} T_i^{\text{gas}} \approx 8,000,000.</script>
<p>If the attacker is targeting transactions within a range of
$(R_\text{lower}, R_\text{upper})$, they can choose
the first $n-1$ transactions to deplete $8,000,000 - R_\text{upper}$ gas
in short steps, and submit $T_n$ to deplete the remaining $R_\text{upper}$
gas with a relatively higher price. Note that the revenue from including a
single transaction is</p>
<script type="math/tex; mode=display">\text{tx_fee} = \text{gas_price} \times \text{gas_usage}.</script>
<p>As gas usage decreases, the
probability of being picked up by miners decreases, so prices should increase
to compensate.</p>
<h1 id="example-fomo3d">Example: Fomo3D</h1>
<p><a href="https://fomo3d.hostedwiki.co/">Fomo3D</a>
is a gambling game where players buy keys from a contract and their money
goes into a pot. At the beginning of each round, a time counter is initiated
which starts counting back from 24 hours. Each bought key adds 30 seconds to the
counter. When the counter hits 0, the last player to have bought a key wins the
majority of the pot and the rest is distributed to others. The
way the pot is distributed depends on the team that the winner belongs to.</p>
<p>Key price increases with increasing key supply, which makes it harder and harder
to buy a key and ensures the round will end after some point. In time, the stakes
increase and the counter reduces to a minimum, like 2 minutes. At this
point, the players pay both high gas and key prices to be “it” and win the game.
Players program bots to buy keys for them, and winning becomes a matter of coding the
right strategy. As you can understand from the subject, the
first round was won through a block stuffing attack.</p>
<p>On August 22 2018, the address
<a class="hex" href="https://etherscan.io/address/0xa169df5ed3363cfc4c92ac96c6c5f2a42fccbf85">0xa16…f85</a>
won
10,469 ETH from the first round by following the strategy I outlined above. The
winner managed to be the last buyer in
block <a href="https://etherscan.io/block/6191896">6191896</a>
and managed to stall
transactions with Fomo3D until block <a href="https://etherscan.io/block/6191909">6191909</a>
for 175 seconds,
ending the round. Some details:</p>
<ul>
<li>
<p>Fomo3D Long contract address:
<a class="hex" href="https://etherscan.io/address/0xA62142888ABa8370742bE823c1782D17A0389Da1">0xA62…Da1</a></p>
</li>
<li>
<p>Last transaction by the winner before the attack starts:
<a class="hex" href="https://etherscan.io/tx/0x7a06d9f11e650fbb2061b320442e26b4a704e1277547e943d73e5b67eb49c349">0x7a0…349</a></p>
</li>
<li>
<p>Transaction ending the round and firing the <code class="highlighter-rouge">onBuyAndDistribute</code> event which
distributes the pot:
<a class="hex" href="https://etherscan.io/tx/0xa143a1ee36e1065c3388440ef7e7b38ed41925ca4799c8a4d429fa3ee1966012">0xa14…012</a></p>
</li>
<li>
<p>Transaction where winner withdraws the prize:
<a class="hex" href="https://etherscan.io/tx/0xe08a519c03cb0aed0e04b33104112d65fa1d3a48cd3aeab65f047b2abce9d508">0xe08…508</a></p>
</li>
<li>
<p>Contract used for block stuffing:
<a class="hex" href="https://etherscan.io/address/0x18e1b664c6a2e88b93c1b71f61cbf76a726b7801">0x18e…801</a></p>
</li>
<li>
<p>Other addresses possibly belonging to the winner:
<a class="hex" href="https://etherscan.io/address/0x3c316def240b1c5ab5e0be027cb91e8ae850ff27">0x3c3…f27</a>
, <a class="hex" href="https://etherscan.io/address/0xc6a4f7f08300a614756f20d823f08417f3ba43e2">0xc6a…3e2</a>
, <a class="hex" href="https://etherscan.io/address/0x81eff1f7a6bf826d550e9503f31e4f2d1973a0ac">0x81e…0ac</a>
, <a class="hex" href="https://etherscan.io/address/0xc96b8de929aea89eb242ed50412da0e4a6e7e590">0xc96…590</a>
, <a class="hex" href="https://etherscan.io/address/0xd27d2afdd35650629c63d4d702eb5f5d2d893642">0xd27…642</a>
, <a class="hex" href="https://etherscan.io/address/0x18da94903de79c18eb0f4f6e6327a4266ef60a9a">0x18d…a9a</a>
, <a class="hex" href="https://etherscan.io/address/0x87c7babe2a9bf81c622a346ffbe57671545854ef">0x87c…4ef</a>
, <a class="hex" href="https://etherscan.io/address/0x9dab207b6c366bb7e1a5d930b865ebda410af0cf">0x9da…0cf</a>
, <a class="hex" href="https://etherscan.io/address/0x7ddae8209776f2c0c94840e3e11c9f3862e2fc4c">0x7dd…c4c</a>
, <a class="hex" href="https://etherscan.io/address/0xb1d4abb185305aad9cda4e3c78fe0faaa9b95aef">0xb1d…aef</a>
, <a class="hex" href="https://etherscan.io/address/0x1a6d5dbb20fd44f04acbaf32712c2dcaaa2bad56">0x1a6…d56</a>
, <a class="hex" href="https://etherscan.io/address/0xf6e89c15731d611a33ddcbf3274cfaaae26e1059">0xf6e…059</a>
, <a class="hex" href="https://etherscan.io/address/0x1dd63481f86b47aa0d429581f1e59db97aa1fa69">0x1dd…a69</a>
, <a class="hex" href="https://etherscan.io/address/0x061d8d0fbceb6754d27eaf0da8facc2f42b0363b">0x061…63b</a>
, <a class="hex" href="https://etherscan.io/address/0x00c463d9e9d276e5689f6bbea54ef248e831e776">0x00c…776</a>
, <a class="hex" href="https://etherscan.io/address/0xa9413093002b8c7139eb6929f53ae760cf397eb8">0xa94…eb8</a>
, <a class="hex" href="https://etherscan.io/address/0xf033ad4b1d63687efc24620be89afd07b21b01f2">0xf03…1f2</a></p>
</li>
<li>
<p>Other contracts possibly deployed by the winner for testing prior to the attack:
<a class="hex" href="https://etherscan.io/address/0xaf1fca749a6625801bb28c93eee763ec346a4eae">0xaf1…eae</a>
, <a class="hex" href="https://etherscan.io/address/0x0c0cee774f02b2f05ad230f1292d8846070255ad">0x0c0…5ad</a>
, <a class="hex" href="https://etherscan.io/address/0x88ef961497f8205d36a53c2e81574fbeae15ed04">0x88e…d04</a>
, <a class="hex" href="https://etherscan.io/address/0x4f427bc21334102943c082ace86b5f3974ab76f8">0x4f4…6f8</a>
, <a class="hex" href="https://etherscan.io/address/0x4870580c874e13b050109c5111e30b06cdc864e5">0x487…4e5</a></p>
</li>
</ul>
<p>The user addresses above were scraped from the Ethereum transaction graph as being
linked to a primary account which supplied them with funds. The contract
addresses were scraped from 0-valued transactions sent from user addresses.
These have a <a href="https://en.wikipedia.org/wiki/Distance_(graph_theory)">distance</a>
of 1, there may be other addresses involved with greater distances.</p>
<style>
table {
font-family: monospace;
font-size: small;
}
.highlighted > td {
background-color: #f6e58d !important;
}
.failed {
text-decoration: line-through;
}
</style>
<p>Below are details of the last 4 blocks preceding the end of the round. The rows
highlighted with yellow are transactions submitted by the attacker. The crossed
out rows are failed transactions. All transactions by the attacker were
submitted with a 501 Gwei gas price, and stuffing a single block costed around 4
ETH. The calls to buy keys generally spend around 300,000~500,000 gas, depending
on which function was called.
Below, you see the successfully stuffed
block 6191906.</p>
<table>
<caption><a href="https://etherscan.io/block/6191906">Block 6191906</a></caption>
<tr>
<th>Idx</th>
<th>From</th>
<th>To</th>
<th>Hash</th>
<th>ETH sent</th>
<th>Gas Price<br />[Gwei]</th>
<th>Gas Limit</th>
<th>Gas Used</th>
<th>ETH spent<br />on gas</th>
</tr>
<tr class="failed highlighted">
<td>0</td>
<td><a href="https://etherscan.io/address/0xF033ad4b1D63687efc24620be89aFD07b21B01f2">0xF03…1f2</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0xb97d8e39db0fab5bc75c72e2fbbf3f1e2651b1e4c3fde5e2db02ee02db3fb8e4">0xb97…8e4</a></td>
<td>0</td>
<td>501.0</td>
<td>4,200,000</td>
<td>4,200,000</td>
<td>2.1042</td>
</tr>
<tr class="failed highlighted">
<td>1</td>
<td><a href="https://etherscan.io/address/0x87C7BABE2A9Bf81C622a346fFbE57671545854eF">0x87C…4eF</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0x96f3038119dcc786820008751380184988e067a3f01c6fbefa177565cc0881b0">0x96f…1b0</a></td>
<td>0</td>
<td>501.0</td>
<td>3,600,000</td>
<td>3,600,000</td>
<td>1.8036</td>
</tr>
<tr class="failed highlighted">
<td>2</td>
<td><a href="https://etherscan.io/address/0xf6E89C15731D611A33DdcbF3274CfAAAe26E1059">0xf6E…059</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0x897fc02810f6bbae8d8bc1a32e1e16f442a4c69fa6f77031e86fe546d6e002b3">0x897…2b3</a></td>
<td>0</td>
<td>501.0</td>
<td>200,000</td>
<td>200,000</td>
<td>0.1002</td>
</tr>
<tr>
<th>Sum</th>
<th></th>
<th></th>
<th></th>
<th>0</th>
<th>1503.01</th>
<th>8,000,000</th>
<th>8,000,000</th>
<th>4.0080</th>
</tr>
</table>
<p>Block 6191907 was a close call for the winner, because their transactions picked
up for the block did not amount up to 8,000,000 and the other transaction was a
call to Fomo3D by an opponent to buy keys. Note that it has a gas price of 5559
Gwei, which means either the bot or person who submitted the transaction was
presumably aware of the attack.
The transaction failed due to low gas limit, presumably due to a miscalculation
by the bot or the person.</p>
<table>
<caption><a href="https://etherscan.io/block/6191907">Block 6191907</a></caption>
<tr>
<th>Idx</th>
<th>From</th>
<th>To</th>
<th>Hash</th>
<th>ETH sent</th>
<th>Gas Price<br />[Gwei]</th>
<th>Gas Limit</th>
<th>Gas Used</th>
<th>ETH spent<br />on gas</th>
</tr>
<tr class="failed">
<td>0</td>
<td><a href="https://etherscan.io/address/0x32AD247B94E46bB75caC37B81e6CB53173002370">0x32A…370</a></td>
<td><a href="https://etherscan.io/address/0xA62142888ABa8370742bE823c1782D17A0389Da1">0xA62…Da1</a></td>
<td><a href="https://etherscan.io/tx/0x5e7309de3aab2a36286fc04aebcfadec627bcb4aa89af4dad82001a5993d1be1">0x5e7…be1</a></td>
<td>0.0056</td>
<td>5559.7</td>
<td>379,000</td>
<td>379,000</td>
<td>2.1071</td>
</tr>
<tr class="failed highlighted">
<td>1</td>
<td><a href="https://etherscan.io/address/0xC6A4F7f08300a614756f20D823f08417F3BA43E2">0xC6A…3E2</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0xb8b11c23da4a4b4600be92fb445868af911836db57e235747e28cbb265f6340c">0xb8b…40c</a></td>
<td>0</td>
<td>501.0</td>
<td>3,900,000</td>
<td>3,900,000</td>
<td>1.9539</td>
</tr>
<tr class="failed highlighted">
<td>2</td>
<td><a href="https://etherscan.io/address/0xD27d2aFdD35650629c63d4D702eb5F5D2d893642">0xD27…642</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0xbcf63a44632366a063ad4d37f7a17c19a95662ebf571d02cfa180d7309032c62">0xbcf…c62</a></td>
<td>0</td>
<td>501.0</td>
<td>3,300,000</td>
<td>3,300,000</td>
<td>1.6533</td>
</tr>
<tr class="failed highlighted">
<td>3</td>
<td><a href="https://etherscan.io/address/0x00c463D9E9D276e5689F6BbeA54eF248e831e776">0x00c…776</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0xf30cc8510c58d34a96e43536e2e1e956f56d3320f8ad16c70a80ec6ac476d337">0xf30…337</a></td>
<td>0</td>
<td>501.0</td>
<td>400,000</td>
<td>400,000</td>
<td>0.2004</td>
</tr>
<tr>
<th>Sum</th>
<th></th>
<th></th>
<th></th>
<th>0.0056</th>
<th>7062.71</th>
<th>7,979,000</th>
<th>7,979,000</th>
<th>5.9147</th>
</tr>
</table>
<p>Transactions in block 6191908 belonged to the attacker except for one irrelevant
transfer. This block is also considered successfully stuffed, since the
7,970,000 gas usage by the attacker leaves no space for a call to buy keys.</p>
<table>
<caption><a href="https://etherscan.io/block/6191908">Block 6191908</a></caption>
<tr>
<th>Idx</th>
<th>From</th>
<th>To</th>
<th>Hash</th>
<th>ETH sent</th>
<th>Gas Price<br />[Gwei]</th>
<th>Gas Limit</th>
<th>Gas Used</th>
<th>ETH spent<br />on gas</th>
</tr>
<tr class="failed highlighted">
<td>0</td>
<td><a href="https://etherscan.io/address/0xD27d2aFdD35650629c63d4D702eb5F5D2d893642">0xD27…642</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0x74acf9d7bb35b8abc2bc3a3bbcd150d59ba5fdbd7303ea657b208ced9138b9b1">0x74a…9b1</a></td>
<td>0</td>
<td>501.0</td>
<td>3,300,000</td>
<td>3,300,000</td>
<td>1.6533</td>
</tr>
<tr class="failed highlighted">
<td>1</td>
<td><a href="https://etherscan.io/address/0x7Ddae8209776F2c0C94840E3E11c9f3862E2fc4c">0x7Dd…c4c</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0x48c720208f47a2d6cd49cc2d539a334ad802732a11233ce1fad61ee686fd2222">0x48c…222</a></td>
<td>0</td>
<td>501.0</td>
<td>2,700,000</td>
<td>2,700,000</td>
<td>1.3527</td>
</tr>
<tr class="failed highlighted">
<td>2</td>
<td><a href="https://etherscan.io/address/0x3C316DEF240b1C5aB5e0be027cb91E8aE850Ff27">0x3C3…f27</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0x01b9d5f0b4b33d47631c1008e5efd296dbe986f1e916f2559253992c430bb4aa">0x01b…4aa</a></td>
<td>0</td>
<td>501.0</td>
<td>1,800,000</td>
<td>1,800,000</td>
<td>0.9018</td>
</tr>
<tr class="failed highlighted">
<td>3</td>
<td><a href="https://etherscan.io/address/0xa9413093002B8C7139EB6929F53aE760CF397eb8">0xa94…eb8</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0x7765ee98603e32ccaef198fc7c495eb360a2a61853d554c55939bd329a947d43">0x776…d43</a></td>
<td>0</td>
<td>501.0</td>
<td>170,000</td>
<td>170,000</td>
<td>0.0851</td>
</tr>
<tr>
<td>4</td>
<td><a href="https://etherscan.io/address/0xbFd35dB76E747C8ed0737954f98B22490F2B31b4">0xbFd…1b4</a></td>
<td><a href="https://etherscan.io/address/0x663C9E9bBF5282bb7485f9D0Cc8Dbc21a0531d31">0x663…d31</a></td>
<td><a href="https://etherscan.io/tx/0x3a650d96b958a2953bfcac0cff9d7aad09463129bb404814e8d42d8e44351ba1">0x3a6…ba1</a></td>
<td>0.05</td>
<td>100.0</td>
<td>21,000</td>
<td>21,000</td>
<td>0.0021</td>
</tr>
<tr>
<th>Sum</th>
<th></th>
<th></th>
<th></th>
<th>0.05</th>
<th>2104.01</th>
<th>7,991,000</th>
<th>7,991,000</th>
<th>3.9950</th>
</tr>
</table>
<p>By block 6191909, the counter has struck zero—more like current UTC time
surpassed the round end variable stored in the contract—and any call to Fomo3D
would be the one to end the round and distribute the pot. And the first transaction
in the block is—wait for it—a call to Fomo3D to buy keys by the opponent
whose transaction failed a few blocks earlier, submitted with 5562 Gwei. So the
guy basically paid 1.7 ETH to declare the attacker the winner!</p>
<table>
<caption><a href="https://etherscan.io/block/6191909">Block 6191909</a></caption>
<tr>
<th>Idx</th>
<th>From</th>
<th>To</th>
<th>Hash</th>
<th>ETH sent</th>
<th>Gas Price<br />[Gwei]</th>
<th>Gas Limit</th>
<th>Gas Used</th>
<th>ETH spent<br />on gas</th>
</tr>
<tr>
<td>0</td>
<td><a href="https://etherscan.io/address/0x32AD247B94E46bB75caC37B81e6CB53173002370">0x32A…370</a></td>
<td><a href="https://etherscan.io/address/0xA62142888ABa8370742bE823c1782D17A0389Da1">0xA62…Da1</a></td>
<td><a href="https://etherscan.io/tx/0xa143a1ee36e1065c3388440ef7e7b38ed41925ca4799c8a4d429fa3ee1966012">0xa14…012</a></td>
<td>0.0056</td>
<td>5562.2</td>
<td>379,000</td>
<td>304,750</td>
<td>1.6950</td>
</tr>
<tr class="highlighted">
<td>1</td>
<td><a href="https://etherscan.io/address/0xC96b8de929aeA89EB242Ed50412da0E4a6e7E590">0xC96…590</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0xf4703d5edf02f2b6fc33c6bdbc442b35f91abf0105132e8bafa9f922d18a59ca">0xf47…9ca</a></td>
<td>0</td>
<td>501.0</td>
<td>2,200,000</td>
<td>37,633</td>
<td>0.0188</td>
</tr>
<tr class="highlighted">
<td>2</td>
<td><a href="https://etherscan.io/address/0xb1D4Abb185305aAd9CDa4e3c78FE0FaaA9b95aEF">0xb1D…aEF</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0xe4c71f694ae2a0b144bd4db938e20b5597d51cbefd3acb643f63abd3e55feedb">0xe4c…edb</a></td>
<td>0</td>
<td>501.0</td>
<td>1,400,000</td>
<td>37,633</td>
<td>0.0188</td>
</tr>
<tr class="highlighted">
<td>3</td>
<td><a href="https://etherscan.io/address/0x18Da94903De79c18Eb0F4F6e6327A4266ef60A9A">0x18D…A9A</a></td>
<td><a href="https://etherscan.io/address/0x18e1B664C6a2E88b93C1b71F61Cbf76a726B7801">0x18e…801</a></td>
<td><a href="https://etherscan.io/tx/0xf3a1a38d631034010ae5449ce14480bad932028bfc34309fb6030b2c31ed8995">0xf3a…995</a></td>
<td>0</td>
<td>501.0</td>
<td>800,000</td>
<td>37,633</td>
<td>0.0188</td>
</tr>
<tr class="">
<td>…</td>
<td>…</td>
<td>…</td>
<td>…</td>
<td>…</td>
<td>…</td>
<td>…</td>
<td>…</td>
<td>…</td>
</tr>
</table>
<p>Another thing to note is that the attacker probably crafted the spender contract
to stop the attack when the round has ended, presumably to cut costs. So the 37,633
gas used by the contract are probably to call the Fomo3D contract to check
round status. All these point out to the fact that the attacker is an
experienced programmer who knows their way around Ethereum.</p>
<p><a href="/assets/fomo3d_round1_tx_details.html">Here</a>, you can see the details of the 100
blocks preceding the end of the round, with the additional information of ABI
calls and events fired in transactions.</p>
<p>Since the end of the first round, 2 more rounds ended with attacks similar to
this one. I didn’t analyze all of them because it’s too much for this post, but
here are some details if you want to do it yourselves.</p>
<div style="overflow-x: scroll;">
<table>
<thead>
<tr>
<th>Round</th>
<th>Address winning the pot</th>
<th>Winner’s last tx before the end of the round</th>
<th>Block containing that tx</th>
<th>Tx ending the round</th>
<th>Block containing that tx</th>
<th>Tx where winner withdraws prize</th>
<th>Amount won [ETH]</th>
<th>Contract used for block stuffing</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>0xa169df5ed3363cfc4c92ac96c6c5f2a42fccbf85</td>
<td>0x7a06d9f11e650fbb2061b320442e26b4a704e1277547e943d73e5b67eb49c349</td>
<td>6191896</td>
<td>0xa143a1ee36e1065c3388440ef7e7b38ed41925ca4799c8a4d429fa3ee1966012</td>
<td>6191909</td>
<td>0xe08a519c03cb0aed0e04b33104112d65fa1d3a48cd3aeab65f047b2abce9d508</td>
<td>10,469</td>
<td>0x18e1b664c6a2e88b93c1b71f61cbf76a726b7801</td>
</tr>
<tr>
<td>2</td>
<td>0x18a0451ea56fd4ff58f59837e9ec30f346ffdca5</td>
<td>0x0437885fa741f93acfdcda9f5a2e673bb16d26dd22dfc4890775efb8a94fb583</td>
<td>6391537</td>
<td>0x87bf726bc60540c6b91cc013b48024a5b8c1431e0847aadecf0e92c56f8f46fd</td>
<td>6391548</td>
<td>0x4da4052d2baffdc9c0b82d628b87d2c76368914e33799032c6966ee8a3c216a0</td>
<td>3,264</td>
<td>0x705203fc06027379681AEf47c08fe679bc4A58e1</td>
</tr>
<tr>
<td>3</td>
<td>0xaf492045962428a15903625B1a9ECF438890eF92</td>
<td>0x88452b56e9aa58b70321ee8d5c9ac762a62509c98d9a29a4d64d6caae49ae757</td>
<td>6507761</td>
<td>0xe6a5a10ec91d12e3fec7e17b0dfbb983e00ffe93d61225735af2e1a8eabde003</td>
<td>6507774</td>
<td>0xd7e70fdf58aca40139246a324e871c84d988cfaff673c9e5f384315c91afa5e4</td>
<td>376</td>
<td>0xdcC655B2665A675B90ED2527354C18596276B0de</td>
</tr>
</tbody>
</table>
</div>
<p>A thing to note in the following rounds is that participation in the game
and amount of pot gradually decreased, presumably owing to the fact that
the way of beating the game has been systematized. Although anyone can attempt
such an attack, knowing how it will be won takes the “fun” factor out of it.</p>
<p>Credit: Although I’ve found previous instances of the term “block stuffing”
online, Nic Carter is the first
one to <a href="https://twitter.com/nic__carter/status/1032262665668894721">use it</a> in
this context.</p>
Thu, 18 Oct 2018 00:00:00 +0000
https://osolmaz.com/2018/10/18/anatomy-block-stuffing/
https://osolmaz.com/2018/10/18/anatomy-block-stuffing/cryptoType Equations up to 2x Faster in LaTeX<p>Typing equations with vanilla LaTeX and amsmath is slow. It involves
lengthy macros and repetition, like typing <code class="highlighter-rouge">\boldsymbol{}</code> every time you want to
print a bold symbol for vectors. Moreover they are hard to remember: Why aren’t
dot and cross product named <code class="highlighter-rouge">\cdot</code> and <code class="highlighter-rouge">\times</code>, and not <code class="highlighter-rouge">\dot</code> and <code class="highlighter-rouge">\cross</code>?</p>
<p>The problem is <strong>context</strong>: We can’t expect amsmath to have context, since there are
many disciplines that use LaTeX to print formulas, and the symbols have
different meaning in different contexts. Amsmath’s job is not to provide
<strong>contextual macros</strong>, but a robust basis for those who’d like to
build them.</p>
<p>We have to build our own macros in order to <strong>type less and express more</strong>. For
example, taking a partial derivative is a very common operation:</p>
<script type="math/tex; mode=display">\frac{\partial x}{\partial y}</script>
<p>In order to print this, I typed</p>
<div class="language-latex highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">$</span><span class="nv">\frac</span><span class="p">{</span><span class="nv">\partial</span><span class="nb"> x</span><span class="p">}{</span><span class="nv">\partial</span><span class="nb"> y</span><span class="p">}$</span>
</code></pre></div></div>
<p>Too many characters for such a simple operation, don’t you think? What if we
defined a macro for partial derivatives:</p>
<div class="language-latex highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">\newcommand</span><span class="p">{</span><span class="k">\partd</span><span class="p">}</span>[2]<span class="p">{</span><span class="k">\partial</span> #1<span class="p">}{</span><span class="k">\partial</span> #2<span class="p">}</span>
</code></pre></div></div>
<p><code class="highlighter-rouge">\partd</code> stands for partial derivative, which is short and easy to remember.
With this macro at hand, I can just type</p>
<div class="language-latex highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">$</span><span class="nv">\partd</span><span class="p">{</span><span class="nb">x</span><span class="p">}{</span><span class="nb">y</span><span class="p">}$</span>
</code></pre></div></div>
<p>The number of characters decreased from 25 to 12, which is more than 50%!</p>
<p>Let’s see what we can do with bold symbols. Amsmath offers <code class="highlighter-rouge">\boldsymbol</code> for
that purpose. To print regular <code class="highlighter-rouge">a</code>, you just type it: <code class="highlighter-rouge">$a$</code>. But to print bold
<code class="highlighter-rouge">a</code>, you have to type 13 more characters: <code class="highlighter-rouge">$\boldsymbol{a}$</code>. This is absurd!</p>
<p>We can define a shorter macro that uses <code class="highlighter-rouge">\boldsymbol</code></p>
<div class="language-latex highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">\newcommand</span><span class="p">{</span><span class="k">\B</span><span class="p">}</span>[1]<span class="p">{</span><span class="k">\boldsymbol</span><span class="p">{</span>#1<span class="p">}}</span>
</code></pre></div></div>
<p>effectively decreasing the number of extra characters from 13 to 4.
But wait! We can do even better.</p>
<p>The only input <code class="highlighter-rouge">\B{}</code> will receive is <code class="highlighter-rouge">A-Z</code>, <code class="highlighter-rouge">a-z</code>, <code class="highlighter-rouge">0-9</code>, and possibly
<code class="highlighter-rouge">\alpha-\omega</code>. So with this limited set of inputs, we can get rid of the curly
braces by defining</p>
<div class="language-latex highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">\newcommand</span><span class="p">{</span><span class="k">\Ba</span><span class="p">}{</span><span class="k">\boldsymbol</span><span class="p">{</span>a<span class="p">}}</span>
<span class="k">\newcommand</span><span class="p">{</span><span class="k">\Bb</span><span class="p">}{</span><span class="k">\boldsymbol</span><span class="p">{</span>b<span class="p">}}</span>
...
<span class="k">\newcommand</span><span class="p">{</span><span class="k">\Bz</span><span class="p">}{</span><span class="k">\boldsymbol</span><span class="p">{</span>z<span class="p">}}</span>
</code></pre></div></div>
<p>and so on for each desired character. This would decrease the number of extra
characters to 2: just <code class="highlighter-rouge">\B</code>!
And we can do this with other math fonts, such as blackboard bold, cursive,
fraktur etc. In fact, I already did that with
<a href="https://github.com/osolmaz/shortsym">shortsym</a>. I also created a
package called <a href="https://github.com/osolmaz/lazyeqn">lazyeqn</a> containing the
contextual macros I always use. You can check both out on my GitHub.</p>
<p>I’ve been using LaTeX since 2010 and enjoy it a lot. I finished homeworks, wrote
my thesis and typed out whole lecture notes, in real time during the lecture.
When I see a friend begin learning LaTeX, I see them lose time setting equations
the hard way, because they don’t know how to define their own macros. I hope
this will help them and others become more efficient.</p>
Fri, 12 Oct 2018 00:00:00 +0000
https://osolmaz.com/2018/10/12/latex-2x-equations/
https://osolmaz.com/2018/10/12/latex-2x-equations/latexprogrammingVisualizing Returns in the Crypto Market<p>This study is a follow-up of the articles
<a href="https://blockchainatberkeley.blog/can-you-beat-the-crypto-market-8d02e91f87af">Can You Beat the Crypto-Market?</a>
and
<a href="https://blockchainatberkeley.blog/can-you-still-beat-the-crypto-market-196255da0dc2">Can You <em>Still</em> Beat the Crypto-Market?</a>
by Anton Muehlemann. Anton attempted to deduce the ROIs for different
types of <a href="https://en.wikipedia.org/wiki/Asset_allocation">asset allocations</a>,
such as investing in large cap or small cap coins, by equal amounts or by
amounts varying with market cap.</p>
<p>Seeing that, I had an idea: what if we discard the data in between and just take
the prices at the days of investment and profit realization. This would give us
the ability to visualize the returns for a <strong>continuous</strong> range of asset
allocations.</p>
<p>Let me clarify with an example: Let’s say we have 1000 coins in
<a href="https://coinmarketcap.com/">CoinMarketCap</a>, sorted by their market
capitalizations. You have $10,000 and
want to invest in a range of coins. You can have different choices
of investment:</p>
<ol>
<li><strong>All of them:</strong> If you were to invest in every coin with an equal asset
allocation, you would have invested $10 to every coin.</li>
<li><strong>Mid 60%:</strong> You invest ~$16.7 to every coin between the ranks 200-800.</li>
<li><strong>Top 20%:</strong> You invest $50 to every coin between the ranks 1-200.</li>
<li><strong>Top 0.1%:</strong> You invest $10,000 to a single coin, which in this case, is
Bitcoin.</li>
</ol>
<p>After a month or a year, your investment would perform differently for these 4
options. A handy tool would be one that can show the performance of these
portfolios and all the values in between. Let’s take a quarter of a circle, and
define a polar coordinate system. Let the radius r denote the amount of coins
invested, and angle θ denote the position of our investment in the total
range of 1000 coins:</p>
<figure>
<p><img src="/img/visualizing_returns/guide.svg" />
Legend for the return slice. 1, 2, 3 and 4 correspond to the different
portfolios given in the example above. To find the ROI when you invest in a
range between a<sup>th</sup> and b<sup>th</sup> coins, follow the reference
lines down from the top arc until they intersect.</p>
</figure>
<p>Now, we have a coordinate system to lay our ROIs. And we don’t have to make a precise
definition of low-cap coins vs high-cap coins. If there is an optimal value in
between, we should be able to see it clearly. Below is an example for January 2017.</p>
<figure>
<p><img src="/img/visualizing_returns/return_matrix_2017-01-01_2017-02-01.png" />
ROIs for January 2017. Value increases are shown by shades of red, whereas
decreases shades of blue.</p>
</figure>
<p>From this figure, we understand that for January, among all 80 of the sufficient
volume coins, the low cap ones were the most profitable. Investing in the bottom
20%—which has 4 of the top 10 increases in value—would have made you more than 50%
profit in one month.
Furthermore, if you open the image in a new tab and zoom in, you would see that
the highest ROI belongs to Decred with 5x, followed by ZCoin with 2.7x.</p>
<p>Notice how high ROI coins paint a triangular region starting from the top. That
region corresponds to allocations containing that coin.</p>
<p>Below is a return slice for the entire 2017.</p>
<figure>
<p><img src="/img/visualizing_returns/return_matrix_2017-01-01_2018-01-01.png" />
ROIs for 2017. The scale was adjusted to take into account the 57x increase for
100% asset allocation.</p>
</figure>
<p>2017 was the year of ICO bubbles, and there was a phenomenal increase in the
capital entering the market. The bubble burst 2 months later, with massive
sellouts reducing the total market cap by ~50%.</p>
<figure>
<p><img src="/img/visualizing_returns/return_matrix_2018-03-01_2018-04-01.png" />
ROIs for March 2018. It’s all blue :(</p>
</figure>
<p>This marked the beginning of a new bear market, and restoration of sanity.</p>
<p><a href="https://ipfs.io/ipfs/QmWx9TwaMKqfZf5ez68WYe92yrStWQUMrotnQepxTdd1N3">Here are the rest of the graphs</a>
I generated for 2015-2018.</p>
<h1 id="conclusion">Conclusion</h1>
<p>I invented this way of visualization to see whether there were a way to detect
patterns and find an optimal range of asset allocation.</p>
<p>I couldn’t find any. Even if there were a pattern to speculative investment, it
wasn’t worth investigating. Many of the increases were due to scams, aggressive
marketing and pump-and-dump groups. The only way to win consistently in this
market is to learn about the tech itself, and get in early when something
promising appears. Following this study, I stopped looking at prices altogether
and started reading.</p>
Wed, 22 Aug 2018 00:00:00 +0000
https://osolmaz.com/2018/08/22/visualizing-returns/
https://osolmaz.com/2018/08/22/visualizing-returns/mathcryptoEnsuring Exactness in Curve-Bonded Dividend Token Systems<p>I have <a href="/math/2018/08/03/token-bonding-1/">written recently</a> on the math
behind curved token bonding, and done some derivations on a recent
dividend token implementation that utilizes it, called PoWH3D. First of all, here is a
<a href="https://medium.com/@optimumregret/the-surreal-madness-of-ethereums-pyramid-schemes-da705fe7d92e">nice post</a>
documenting the failure of its previous iteration.</p>
<p>Despite its creators’ marketing it as a gambling ecosystem, the math behind
PoWH3D might prove to be a significant contribution to creating a reliable
dividend-based <a href="https://en.wikipedia.org/wiki/Basic_income">UBI</a> system. Because
unlike its predecessor which failed miserably due to math and implementation
errors, it works (at least as of writing this). Like clockwork, 10% of every
buy, sell and transfer goes to token holders.
More than 70,000 ETH (~$30m)
were deposited to the contract at the all time high.</p>
<p><em>But the contract is not perfect.</em> Let’s entertain the following scenario:</p>
<blockquote>
<p>Nobody has bought P3D yet, and you send 10,000 ETH to the contract. 9000 ETH
worth of P3D are minted, and 1000 ETH is distributed to the only token holder,
you. Then you decide to sell the tokens you have just bought. Their worth in
ETH is computed, and you receive 9/10th of it. The dividends from the sale go
to the only token holder—again, you. So after the purchase and sale, you only
have a fraction of your original 10,000 ETH, and the rest is stored in the
contract as dividends, belonging to you. So withdrawing the dividends, you
should end up with 10,000 ETH, right?</p>
</blockquote>
<p>I tested the scenario and,
<a href="https://github.com/osolmaz/powh3d-study/blob/master/test_exact_dividends.py">here is the code</a>
to replicate the results. Below is the output.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Change in own ETH balance: -885.0215718635744
P3D balance: 0.0
Contract ETH balance: 900.0215718635744
Contract P3D supply: 0.0
</code></pre></div></div>
<p>All P3D has been burned and dividends withdrawn, but 900 ETH remain in the
contract, essentially bricked. This corresponds to ~9% of the initial
investment. Why is that?</p>
<p>In a purchase or sale, the relationship between minted/burned tokens and
spent/reimbursed ethers is nonlinear, expressed by a non-constant supply-price
relationship:</p>
<figure>
<p><img src="/img/token_bonding/supply_price.svg" /></p>
</figure>
<p>And the dividends correspond to 9/10th of that area. The problem is how the
dividends are distributed. Your share is computed from your token balance
<em>after</em> the transaction. So if you sell all your tokens at once, your share of the
dividends from that sale is zero.</p>
<p>If one intends to sell large amount of tokens, this can be mitigated by dividing
the sale into installments. The amount of bricked ETH decreases exponentially
with increasing number of installments, as tested
<a href="https://github.com/osolmaz/powh3d-study/blob/master/installment_sale.py">in this code</a>.</p>
<figure>
<p><img src="/img/exact_dividends/bricked_eth_installments.svg" />
Percentage of bricked ETH for the scenario above, for token sales consisting of
different number of installments.</p>
</figure>
<p>The computation of the contract becomes exact for an infinite number of
installments, given the transactions are costless. The same problem exists for
purchases. <em>Buyers are overpaid and sellers are underpaid in dividends,
and the effects don’t cancel out for large sums, as demonstrated above.</em></p>
<p>So how can we design this system to be exact? The computation of dividends needs
to account for the varying supply-price relationship.</p>
<h1 id="purchase-dividends">Purchase Dividends</h1>
<p>Let $S_0$ be the initial token supply, $\bar{T}_0$ our initial token balance,
and $R_d$ the dividend ratio of the system, e.g. 1/10 for PoWH3D.
Let us send $E$ ETH to the contract and received $T$ tokens in return, where
$D=R_d E$ is supposed to be
distributed as dividends. In PoWH3D, our dividends $\bar{D}$ from this purchase
is computed as</p>
<script type="math/tex; mode=display">\bar{D} = \frac{\bar{T}_0+\delta T}{S_0+\delta T} D.</script>
<p>Let us define the varying supply $S$ a function of the initial supply $S_0$ and
token increment $\delta T$:</p>
<script type="math/tex; mode=display">S := S_0 + \delta T.</script>
<p>Using this, we can formulate the equation above in terms of $S$ instead of
$\delta T$:</p>
<script type="math/tex; mode=display">\bar{D} = \left[1-\frac{S_0-T_0}{S}\right]D \tag{$\ast$}</script>
<p>and the sum of dividends distributed to everyone else can be computed as</p>
<script type="math/tex; mode=display">\tilde{D} := D-\bar{D} = \frac{S_0-T_0}{S}D.\tag{$\ast\ast$}</script>
<p>However, as demonstrated above, this computation yields inexact results. To
achieve exact results, we have to integrate over the $S$-$P$ curve. The
relationship between spent ether and received tokens is</p>
<script type="math/tex; mode=display">E = \int_{S_0}^{S_0+T} P\, dS</script>
<p>For a linear bonding curve</p>
<script type="math/tex; mode=display">P(S) = P_0 + S I_p,</script>
<p>we have $E$ and $T$ in terms of each other:</p>
<script type="math/tex; mode=display">E(S, T) = T P_0 + T I_p S + \frac{1}{2} T^2 I_p</script>
<p>and</p>
<script type="math/tex; mode=display">T(S, E) = \frac{\sqrt{S^2I_p^2 + 2E I_p + 2 S P_0 I_p + P_0^2}-P_0}{I_p} - S.\tag{$\dagger$}</script>
<p>Deriving from $(\ast)$, our <em>exact</em> dividend from the sale is calculated as</p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{aligned}
\bar{D} &= \int_{S_0}^{S_0+T} P R_d \left[1-\frac{S_0-T_0}{S}\right] \, dS \\
&= \frac{R_d}{2}\left[
I_pT^2+2I_pTT_0+2P_0T + 2P_0(S_0+T_0)\ln\left(\frac{S_0}{S_0+T}\right)
\right]
.
\end{aligned} %]]></script>
<p>Calculation of the <em>exact</em> dividend of everyone else follows similarly
using $(\ast\ast)$</p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{aligned}
\tilde{D} &= \int_{S_0}^{S_0+T} P R_d \left[\frac{S_0-T_0}{S}\right] \, dS \\
&= R_d (S_0-T_0) \left[I_p T - P_0 \ln\left(\frac{S_0}{S_0+T}\right)\right].
\end{aligned} %]]></script>
<p>Summing these up and substituting $(\dagger)$, we see</p>
<script type="math/tex; mode=display">\bar{D}+\tilde{D} = R_d E = D</script>
<p>which verifies the results.</p>
<p>Similar to PoWH3D, $\tilde{D}$ (sum of the dividends of everyone except the
purchaser) is distributed using token balances as weights. The difference is
that the dividends of the purchaser and the rest are computed separately.</p>
<h1 id="sale-dividends">Sale Dividends</h1>
<p>For a sale, the relationship between reimbursed ether and received tokens is</p>
<script type="math/tex; mode=display">E = \int_{S_0-T}^{S_0} P\, dS</script>
<p>For a linear curve, we have the following relationships of $E$ and $T$:</p>
<script type="math/tex; mode=display">E(S, T) = T P_0 + T I_p S - \frac{1}{2} T^2 I_p</script>
<p>and</p>
<script type="math/tex; mode=display">T(S, E) = \frac{P_0-\sqrt{S^2I_p^2 - 2E I_p + 2 S P_0 I_p + P_0^2}}{I_p} + S.</script>
<p>This time, the dividends are calculated according to</p>
<script type="math/tex; mode=display">\bar{D} = \frac{\bar{T}_0 - \delta T}{S_0 - \delta T} D</script>
<p>and the varying supply is calculated as</p>
<script type="math/tex; mode=display">S := S_0 - \delta T.</script>
<p>Substituting, we see that $(\ast)$ and $(\ast\ast)$ also hold for sale.
For our dividends, we have</p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{aligned}
\bar{D} &= \int_{S_0-T}^{S_0} P R_d \left[1-\frac{S_0-T_0}{S}\right] \, dS \\
&= \frac{R_d}{2}\left[
-I_pT^2+2I_pTT_0+2P_0T - 2P_0(S_0-T_0)\ln\left(\frac{S_0}{S_0-T}\right)
\right]
.
\end{aligned}
. %]]></script>
<p>For the rest, we have</p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{aligned}
\tilde{D} &= \int_{S_0-T}^{S_0} P R_d \left[\frac{S_0-T_0}{S}\right] \, dS \\
&= R_d (S_0-T_0) \left[I_p T + P_0 \ln\left(\frac{S_0}{S_0-T}\right)\right]
.
\end{aligned} %]]></script>
<h1 id="conclusion">Conclusion</h1>
<p>I discovered an inconsistency in a smart contract worth $30m while
playing with it. Although it may not be critical, the case proves that people
are OK with pouring money into a project that hasn’t even released a whitepaper,
as long as they are half-sure that they will profit from it. My intuition
makes me doubt that this can allow someone to exploit the contract. But who
knows; if someone manages to hack it, the prize is at least 10 times bigger than
it was in PoWH.</p>
<p>PoWH3D aside, the formulation above can be used to achieve exactness in any
dividend token. I didn’t go into how it can be integrated into a smart contract,
because it is out of the scope of this study. The right way of keeping account
of dividends is not having a map of addresses to dividends and iterating that
map in every transaction, but keeping track of profits per token, which achieves
the same result implicitly. This is how PoWH3D does it, but since my concerns are
theoretical, I didn’t delve into how this might be implemented.</p>
Thu, 16 Aug 2018 00:00:00 +0000
https://osolmaz.com/2018/08/16/exact-dividends/
https://osolmaz.com/2018/08/16/exact-dividends/matheconomicscryptoDangling Words<p>There are words in the English language that I feel majority of speakers do not
understand but are kept in use nonetheless: <em>exacerbate, succinct, plethora,
ambivalent, etc.</em> These words catch my attention, because I don’t understand
them or remember a time I didn’t. The feeling is especially strong when I learn
there is an easy-to-decipher alternative with the same meaning that
people choose not to use. <em>To exacerbate</em> means <em>to worsen</em>
for example.</p>
<p>Except that it doesn’t. In Latin, <em>to exacerbate</em> literally means <em>to make bitter</em>, a
metaphor for making things worse, which got forgotten when it was adopted in
English. If you were to use the word on a
Latin-speaker who hasn’t heard it before, they would be able to deduce the
meaning since they would know <em>acerbus</em> means <em>bitter</em>. But if you were to
eliminate the word from every writing
and dictionary of the Anglosphere now, <em>exacerbate</em> would cease to exist in a
100 years. That’s because it
doesn’t share a connection with any other word; it’s dangling out in the open.</p>
<p>This is not a rant against loanwords and foreign elements in language;
<a href="https://de.wikipedia.org/wiki/Deutscher_Sprachpurismus">many</a>
<a href="https://en.wikipedia.org/wiki/Linguistic_purism_in_Korean">tried</a>
<a href="https://en.wikipedia.org/wiki/Turkish_Language_Association">that</a>
<a href="https://en.wikipedia.org/wiki/Tanittamil_Iyakkam">before</a>
<a href="https://en.wikipedia.org/wiki/Hela_Havula">and</a>
<a href="https://en.wikipedia.org/wiki/Høgnorsk">achieved</a>
<a href="https://en.wikipedia.org/wiki/Hindi–Urdu_controversy">success</a>
<a href="https://en.wikipedia.org/wiki/Accademia_della_Crusca">to</a>
<a href="https://en.wikipedia.org/wiki/Linguistic_purism_in_Icelandic">a</a>
<a href="https://en.wikipedia.org/wiki/Katharevousa">certain</a>
<a href="https://en.wikipedia.org/wiki/Linguistic_purism_in_English">degree</a>.
Linguistic purism in the industrial age have mostly been motivated by
nationalism and identity politics. My concerns, which motivated me to write this
are based on comprehension and communication efficiency.</p>
<p>When used, dangling words evoke the need to shout out
<a href="https://youtu.be/X8guAnshoMk?t=34">“Speak English!”</a>.
Instead of contributing meaning, they confuse the listener. This may be
strategic when a speaker wants to misdirect his/her audience from what is
being said, which may be something disagreeable or nothing of value. Many
people use obscure words as leverage in a conversation, to confuse and dominate
the listener. Practitioners of law can
<a href="https://en.wikipedia.org/wiki/Legal_English">do that</a>
very easily, for example.</p>
<p>My purpose for writing this is not to talk you out of using dangling words; I
used some myself up until this point. I want to give you they key to
unlock the meaning behind these words: etymology.</p>
<p>I am not a native speaker of English but Turkish. But I had the same problem in
both languages: not being able to remember some words by their definitions in
the dictionary. I used to write them down many times, but I had trouble
remembering in school. I had a hard time breaking down the
words into concepts. Yes, I understood that <em>exacerbate</em> meant to <em>make worse</em>,
but I didn’t understand how exactly: did <em>exacer</em> mean make and <em>bate</em> worse? I
couldn’t associate conceptually, namely tie those imaginary strings
between words. How was I supposed to remember, and how did everybody else manage
to do it?</p>
<p>Years later, when I discovered etymological dictionaries, I found my answer: I
had been taught wrong all along. In computer science terms, what my teachers
wanted me to do was to create a new slot for every word in an
<a href="https://en.wikipedia.org/wiki/Array_data_structure">array</a>.
What I’ve been yearning to do was to create a
<a href="https://en.wikipedia.org/wiki/Graph_(discrete_mathematics)">graph</a>, because
deep down I found it impractical to remember without making any connections.
And those dangling words: they were the most difficult ones because they didn’t belong
to my graph.</p>
<p>With Latin and Greek words, the problem was that I didn’t have the necessary
software to read and write to that graph; I hadn’t learned the basics of
grammar. Fortunately I didn’t have to learn all of the grammar, but only the
part related to word-building. So I learned the few
<a href="https://en.wiktionary.org/wiki/Category:Latin_prefixes">prefixes</a>,
<a href="https://en.wiktionary.org/wiki/Category:Latin_suffixes">suffixes</a> and root
words dictionaries had to offer and bam! I was able to guess the meaning
<a href="https://en.wikipedia.org/wiki/List_of_Latin_words_with_English_derivatives">Latin</a>
and
<a href="https://en.wiktionary.org/wiki/Appendix:Ancient_Greek_words_with_English_derivatives">Greek</a>
words I haven’t heard before.</p>
<p>For example <em>ascend</em>, <em>descend</em>, <em>condescend</em> and <em>transcend</em> are connected because
they come from the same root, meaning <em>to climb</em>. But <em>exacerbate</em> is still
dangling because there is not a loanword that shares the same root. The
conclusion is that there are different levels of dangling:</p>
<ol>
<li><strong>Not dangling:</strong> Words’ connection can be deduced without knowledge of
foreign grammar. Examples: <em>bold</em> and <em>embolden</em>, <em>smart</em> and <em>to outsmart</em>.</li>
<li><strong>Semi-dangling:</strong> Words’ connection can be deduced with knowledge of
foreign grammar, root words and affixes. Examples: <em>art</em> and <em>artificial</em>,
<em>impress</em> and <em>express</em>.</li>
<li><strong>Dangling:</strong> There is no connection to another word. Examples: <em>exacerbate</em>,
<em>myriad</em>, <em>ephemeral</em>, <em>fulcrum</em>.</li>
</ol>
<p>This also implies that compound words are easier to remember.</p>
<p>Learning etymological roots</p>
<ul>
<li>improved my writing and understanding of English,</li>
<li>kicked up my bullshit detector a few notches,</li>
<li>increased my ability to assess a person from the words they use,</li>
<li>and gave me immunity against technostrategic language.</li>
</ul>
<p>But this was English, which is not my native language. I still wasn’t feeling
smart enough in Turkish, so I had to repeat the same procedure for Arabic and
Persian, which are to Turkish what Latin and Greek are to English. But there was a
problem: Despite there being
<a href="http://www.nisanyansozluk.com">Turkish etymological dictionaries</a>, there was
no modern book for the grammar of Arabic and Persian loanwords. There are
<a href="http://akadan.anadolu.edu.tr/sites/akadan.anadolu.edu.tr/files/files/osmanli_turkcesi_I.pdf">accessible books of “Ottoman Turkish”</a>,
but you have to learn the old script and grammatical terminology in
order to understand them.</p>
<p>I did that and it took me a few months, but its results will last a lifetime:
I feel competent at speaking, reading and writing my native language.
I taught myself the Turkish my teachers failed to teach.</p>
<p>It was bothering me that there was no modern book on the grammar of
loanwords. Almost all of the conceptual words neologized in the pre-republic era are
built from Arabic, accounting for 90% of the words that
confused me.
For example, the vocabulary legal Turkish consists mainly of
Arabic loanwords and neologisms native to Turkish.
So I mainly focused on Arabic.</p>
<p>First, I created a
<a href="http://osolmaz.com/misc/arapca-turetme-tablosu">web app</a>
for compounding Arabic roots.
Then I scraped an etymological dictionary and manually classified all Arabic words
according to their declensions into a data file. Using that data file, I wrote a
<a href="https://github.com/osolmaz/devsirme-kelimeler-sozlugu">book</a>
containing a modernized grammar of Arabic loanwords and dictionaries mapping
between different roots and their declensions. This helped me to connect
90% of the words I knew. My mind map was finally complete.</p>
<p>I wasn’t satisfied with the way I was taught how to read and write, so I hacked
my brain to make it right. At
school, they teach you that language is this rigid thing you
can’t change and there is only one right way of thinking and writing. Like us,
language is fluid and shaped by everything we do; dictionaries and grammar books
are futile attempts at tying up something that can’t be tied up.
I found my own way to interpret it and undangled my words. Now,
the world makes more sense.</p>
Mon, 13 Aug 2018 00:00:00 +0000
https://osolmaz.com/2018/08/13/dangling-words/
https://osolmaz.com/2018/08/13/dangling-words/language