good progress on pytorch game today

cool use case of llms is indexing/searching through files

September 8, 2023

need to automate all schoolwork

todos for homework bot:

for pytorch game, could write script that takes in pytorch doc link, and creates question based on it

can also add chat so user could ask questions specifically about that torch method

need to become a gwern type mf

September 6, 2023

todos for homework project:

might just work on pytorch game this morning though

September 5, 2023

has anyone made a general use Photomath

could be a fun way to get into computer vision

could have a couple different use cases:

September 4, 2023

maybe pytorch game is better as a companion to the docs

not sure how to turn it from tutorial to game

policy network is not converging

September 1, 2023

gonna get first training run done tonight

todos:

August 29, 2023

need a way to tell model whose turn it is

otherwise it might generalize to a move suitable for both

can just make first dimension current player's stones, and other one is opponent

instead of 0 = white, 1 = black

August 28, 2023

made cool visualization for go game

just demonstration that I am getting correct board state at any given move

should be able to start on policy network tomorrow, since training data is in correct format now

August 27, 2023

August 26, 2023

todos:

finished zero to one

absolute banger

reading log is deployed

need to figure out whether new home page is good idea or not

published to website, may change in the future

August 25, 2023

too tired to do anything useful

gonna build reading list rn

August 24, 2023

first step of go bot is policy network (just CNN trained on pro games)

should be relatively straightforward

good paper to base policy network on

could be cool to look at what different layers of cnn will capture

this paper addresses symmetry of the board in a cool way

“All Rhodes Scholars had a great future in their past”

zero to one is really good so far

August 23, 2023

despite being a little verbose, there is probably some alpha still on LessWrong

lots of value in understanding bayesian thinking and rationality

even if just to understand doomer args

really need to read SICP

have had the book for months now

no better time than the present

gonna get reading list up as a way to apply social pressure

ok im making new home page

done with new home page

not gonna push it till reading log is done

August 21, 2023

there are so many things to learn

need to read more about

coming up with smart/interesting ideas seems to work the same as coming up with good startups according to pg

first you need to get to the edge of innovation, then look for holes

to have interesting commentary about politics/culture/economics, you first need to grind your way to the edge

read more books, use twitter/instagram/distractions less

become a well-read autodidact builder

things to be working on:

first gonna make a go bot instead of poker

might be harder to make it really good, but is much more straightforward due to complete information

would also be cool to add to my website

seems like only new topic is tree search

saying that like it is trivial

August 19, 2023

for poker model, will probably have to start with heads up(only 1 other player)

need to encode all information about a hand, including bets up to this point

betting pre-flop vs post-flop are very different

basic probabilities are easy to compute

information required per hand:

things to decide:

data might be hard to find

slightly sketchy dataset:

August 18, 2023

could keep randomness by sending two functions to run, that both compute answer

has anyone made poker bots

that they could then use for online games

this doesn’t seem that hard to build

problem with poker model is that you don't want to recreate average player

(what you'd get by training on all games)

instead, need some kind of decision tree like alpha go

might work if you get a bunch of pro games

but they would be playing against other pros, which you don't playing against other players

maybe v1 is just on pro games

later version could be alphaGo-style decision tree

August 16, 2023

I have gotten nothing done in the past ~2 weeks

Need to build/read/learn

August 9, 2023

who is making decentralized gpu compute

also would be fun to check out smallville agents

Need to update site

Move current one to ‘old’ subdomain

Maybe just an html page? Looks more aesthetic + faster load time

react is too bloated

Todos:

reading log page

new homepage

write levels for pytorch game

create three animation for game

Refactor

Want it to look something like this

become an autodidact

August 8, 2023

solution is serverless functions(aws lambda)

making solid progress

can run pytorch code and get output

todo:

August 7, 2023

once set up at school ill resume the music generation development

for now just gonna work on pytorch game

gonna take physics or chem next semester

need to become more versed in hard sciences

CS classes are likely going to be fairly straightforward until sophomore year

problem is that python-shell cant install torch on every run

probably gonna have to set up instance that will run the code😔

August 3, 2023

music gen from relative attention is really inconsistent

half the time it sounds amazing, half the time it plays 400 notes in 3s

last day at internship

i need more compute

didn't realize that lambda labs is constantly out of GPUs

i need a way to figure out if the model is bad because it is a bad architecture or just if the model is too small

hoping its the former

fun tool would be coding puzzles you solve with pytorch

has anyone made that?

i.e. beginner puzzle would be make tensor of this shape that has these properties

might be hard to make more complicated puzzles though

complicated levels would be to get a model to a certain loss

something like kaggle but more game-like

codeMirror as editor

python-shell to run code on nodejs

should be a game though, not a test

something like this:

August 2, 2023

maybe I should do electrical or computer engineering

i need more hard science knowledge

maybe ill just take a bunch of physics/chem classes for fun

i have basically no understanding of electricity/energy

finished virtual assistant project yesterday

want to add visual aspect though(~wav2lip)

want to do something with tweet data

something like this:

visualizations are always fun to make

August 1, 2023

still think midi is the way to go for music

actual audio files don't have enough structure

was going to train model on cloud GPUs, but lambda had zero left

they are supposed to be the cheapest by far

still ~$2/hr though

todos:

July 31, 2023

July 30, 2023

fine tunining did not work

base model probably massively overfit

todos:

relative attention is implemented

going to try using cloud gpu tomorrow

also found a good tts model

July 29, 2023

really wish there was linear algebra in high school

sometimes it feels like all of my opinions are just conglomerations of things i've heard

i know that is how learning works, but sometimes it still feels fraudulent