What can be extracted from text, besides sentiment

Anton Kolonin28 Oct 2023

Problem

A team wants a leading signal for a market metric from the news and social media text it already collects. Off-the-shelf sentiment scores help little: the target is numeric, so there are no labels and no accuracy to compute, and about half of the standard models correlate with price at about zero. Thousands of text features cannot be screened by eye, and a brute-force search finds a "predictor" even in noise.

Example from the article: SingularityDAO, an AI trading project, took about 20 open sentiment models and about 200 crypto feeds from Twitter and Reddit. It needed a signal that moves before Bitcoin trading volume and price.

Solution in brief

Compute sentiment and cognitive distortion counters for every feed, and sum them into composite curves. Shift each curve back in time and keep it only if the correlation peaks at shift -1 against the neighbouring shifts. Cognitive distortions alone carried the signal, and sentiment added almost nothing. The talk shows this only in-sample and in a simulated trading backtest.

Questions this case answers
  • How to build a leading indicator from unstructured text for a numeric target when the text has no labels and the target has no classes
  • How to check that a text-derived signal leads the target instead of lagging it after searching thousands of candidate series
  • How to build leading indicators from social media text for forecasting a market metric
  • How to choose a sentiment model when there are no labels, only a numeric target such as price

Tools and methods

  • Aigents sentiment model — interpretable n-gram dictionary model with four metrics (P, N, P+N, contradictive).
  • Cognitive distortion counters — 12–14 dictionary features from the PNAS paper.
  • Composite indicators with shift — sum or product of features, correlation against the target at shifts from -10 to 10.
  • Backtesting — 12 market-making strategies in a simulator that imitates Binance.
Continue reading — it's free

Google or a one-time email code. No password.