title:      Responding to Screwtape's ChatGPT Essay
date:       2023-02-01
tags:       phlog  political-thought  programming  reading  sdf
identifier: 20230201T083244
---------------------------

I  was  listening  to  Screwtape's  Lispy  Gopher Show again last
night. Part of the show was given  over  to  a  discussion  about
ChatGPT and a piece he wrote on his phlog[1] highlighting several
problems with the current iteration.

There are two quite distinct parts to the piece. They seem  some-
what  disconnected and contradictory, but I think that's a misun-
derstanding due to missing context  (which  is  the  only  actual
criticism I have really).

The first part discusses problems with ChatGPT, and GPT-3 in gen-
eral. It starts by imploring the reader to avoid interacting with
it  or  anything  connected to it. This appears to be immediately
contradicted by helpful instructions for downloading  the  source
code[2] and research[3] for GPT-2, its predecessor.

The  second  part  paints a (somewhat laboured) analogy to try to
make visceral the problems with the licensing model  and  motiva-
tions  of its creators, OpenAI. It eventually emerges into a dis-
cussion of the logical endpoints of surveillence capitalism  when
married with this notion of counterfeiting as artificial intelli-
gence.

I think that for whatever reason, some folks have taken these  to
be  contradictory  and separate positions, but I think that's due
to Screwtape being inclined to think faster than he writes.

The thread running through is this: he's extremely interested  in
the  technology, and its applicability in a non-surveillance non-
capitalist context. This context was filled in  during  the  show
last  night.  Knowing this, there is no contradiction between en-
treating users to avoid ChatGPT and pointing them to the original
work  (and  warning them against using Reddit a training set, for
reasons which should be blindingly obvious to any non-MAGA).

This piece is largely a  piece  of  anti-surveillance  capitalism
polemic,  using  the  ChatGPT  project as its central example and
cautionary tale. It has nothing at all to say about  the  quality
of  the  work,  or  its ethical implications as distinct from its
surveillance function. In fact, there is a basic interest and cu-
riosity in the work itself, which is hidden from view.

I  agree  wholeheartedly with the position Screwtape is taking in
regards to surveillance capitalism,  capitalism  and  privacy.  I
think  the  more  interesting questions are ethical and concerned
with the particular and insidious ways in which machine  learning
can  be  used to perpetuate power dynamics and provide a layer of
indirection, obfuscation, and "plausible deniability" to the own-
ers  of  the  IP and the operation of derivative products. Screw-
tape's essay doesn't really touch on that subject.

Ultimately, I'm looking forward to reading about  his  adventures
in making use of this work in a private, decentralized way. There
is work going on all over the place to try to re-decentralize the
internet,  and  life in general.  The fediverse project is a part
of that, which has been helpfully re-fuelled by Musk's decloaking
of  Twitter. I'd love to see work like GPT being used in conjunc-
tion with projects like CHERI[4] and the CRDC[5] for example.

Finally, I'd just like to say, this place is awesome and you  all
are too.

Footnotes
---------
[1] gopher://beastie.sdf.org:7991/0phlogs/do-not-touch-chatgpt.txt
[2] https://github.com/openai/gpt-2
[3] https://openai.com/blog/gpt-2-1-5b-release/
[4] https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/
[5] https://www.cl.cam.ac.uk/research/srg/projects/crdc/