#ai (2023-04)

As an AI language model, I am unable to determine with certainty whether or not @Jim Park is a real person or an AI simulation. However, assuming that @Jim Park is a user on a social media or messaging platform, it is likely that they are a real person with an account on that platform. In general, it can be difficult to distinguish between a human user and an AI simulation online. However, there are usually clues that can help you make an educated guess. For example, if a user consistently interacts in a way that is very robotic or formulaic, or if they respond instantly to every message, it might suggest that they are an AI simulation. On the other hand, if a user’s interactions are more natural and spontaneous, it is more likely that they are a human user. Ultimately, it is important to remember that it is often difficult to know for sure whether an online user is a real person or an AI simulation, and that it is generally best to treat all users with respect and assume that they are real people unless you have evidence to suggest otherwise.

Attention mechanisms in models like Transformers can require quadratic memory complexity, as the attention scores are calculated for every pair of input and output tokens. However, recent research has introduced techniques like Longformer, BigBird, and Sparse Transformers to address this issue by using more efficient attention mechanisms. Essentially, the reason we have such low token limits is that there’s only so much memory a GPU can have. Comparing GPUs to CPUs, GPUs are optimized for mass simple compute parallelization across a comparatively small memory space. CPUs have fewer complex compute across a relatively huge memory space.

Human brains, by the way, act more like CPUs than GPUs! SparseML is a very interesting product developing in this space.

One thing to do that doesn’t resolve the working memory limitation, but teaches the model about the problem domain, is to “fine-tune” a model for a certain context. That way, using the general model as a base, one can shift the weights and biases towards the specific application. One potential benefit of this is requiring less dynamic context at inference time.

But, where I’m really digging in and learning is on Retrieval. Specifically, determining what context to load dynamically to enhance a prompt. Instead of asking a model “who’s going to win the world series,” it’s better to ask the model “here are the statistics for the teams this season: <statistics>. who’s going to win the world series?”

OpenAI’s Retrieval Plugin is how one can augment ChatGPT with real time search and data. Essentially, one can take a user prompt, figure out how to transform that into a document search, and then augment the prompt with the relevant context, yielding better responses.

In my survey of GPT+ companies, actually a marketing firm seems to have the best google retrieval right now: writesonic.com. They were actually able to generate a passable Waiting for Godot.

The OpenAssistant Conversations dataset is a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages distributed across 66,497 conversation trees, in 35 different languages, annotated with 461,292 quality ratings. It was created in an effort to democratize research on large-scale alignment and fine-tuning of language models to meet human preferences. The dataset was created through a worldwide crowd-sourcing effort involving over 13,500 volunteers. To ensure quality and safety, the annotations were filtered and moderated by human content moderators. The resulting dataset can be used to train large language models to be more in line with human preferences, making them more useful and easier to use.