mastodon 4.7.3Computer Science Professor at Northeastern. Believes AI should be transparent. http://baulab.info
I want to show the NSF there would be broad support+utility for a "National Deep Inference" service for >100b LLMs.
If your research would be enabled by an inference service on open LLMs w API access+overrides to internal activations, params, gradients: please boost this thread!
(I'm also gathering feedback on twitter - more details here:)
I have put together a curated list of about 50 papers that (IMO) were seminal in defining what "Deep Learning" is today, to share with my students. Goes from 1943-2022.
https://papers.baulab.info/00_README.html
Question for the AI fediverse: What is missing off this list that should be on it?
Had a great time meeting people at #BlackboxNLP and #EMNLP2022 last week. You can see my talk as well as great talks by Lena Voita and Catherine Olsson on youtube here:
OK, I get it now: the "Federated" feed shows all posts that people on my local server follow
I am switching to sigmoid.social. The federated feed here is way more interesting to me than my original random server.
@geomblog@mastodon.social Thanks! Emailing a few qs!
I have tried the first few steps of the prompts in this article.
I can confirm that GPT can define a language along with conjugations and do basic translations applying the words and rules that are set out.
Ok, wow.
My student Rohit Gandikota is finding that large diffusion models like SDXL can be controlled very precisely with low-rank changes in parameters.
His twitter thread on "Concept Sliders" is also a great survey of current diffusion model controllability work. The nice thing about modern image synthesis is that the results are gorgeous.
Help him spread the word!
https://x.com/RohitGandikota/status/1727410973638852957?s=20
My student Koyena Pal will be presenting some cool work at #CoNLL2023.
Her Future Lens (https://future.baulab.info) can look a single hidden state of an LLM and see what a transformer is planning, several tokens ahead.
It can make reading the internal states much more intuitive!
Look for us if you'll be in Singapore for EMNLP/BlackboxNLP/CoNLL.
Work with Jiuding Sun, Andrew Yuan, and Byron Wallace.
https://twitter.com/kpal_koyena/status/1723026282781581380?s=20
Eric Todd has a really interesting new preprint on arXiv:
He shows LLMs contain vector representations of functions that compose and apply in diverse contexts.
It could be a powerful tool for understanding reasoning mechanisms within LLMs.
Eric explains more on a twitter thread here
https://x.com/ericwtodd/status/1717277426873766104?s=20
It's his paper of his PhD. Help him spread the word!
@tdietterich@mastodon.social @aaronhertzmann@fediscience.org
People are speculating, "it's over." If so: what a strange way to close down a major property.