Books that operate in a narrative/temporal style are hard to manage, because statements of fact are mutative.
Consider the story:
"Justin is hungry. Justin eats dinner. Justin is not hungry."
You ask the chatbot "Is Justin hungry?". There is a temporal aspect to this question that is hard for simple systems that are just embedding facts into a vector DB (or similar techniques) to reconcile.
Me:
Consider the story: "Justin is hungry. Justin eats dinner. Justin is not hungry."
Is Justin hungry?
ChatGPT:
No, Justin is not hungry after eating dinner.
The example was to just illustrate the general problem. Think of ingesting a whole novel that takes place over a few years. The whole novel doesn't fit into GPT's context window (which is only a page or two of text). So you have to extract individual statements of fact and index over them (e.g. with semantic indexing, or many other techniques).
It's tricky to deal with cases where the state of something changes many times over the course of the years in the novel.
Imagine you ingest the whole Harry Potter series. You ask the chatbot "How old is Harry Potter?". The answer to the question depends on which part of the story you are talking about. "Does Harry know the foobaricus spell?" The answer depends on which part of the story you are talking about.
Whereas for a non-fiction book typically does not contain these temporally changing aspects. In a book about astronomy, Mars is the 4th planet from the sun in chapter 1, and in chapter 10.
> Think of ingesting a whole novel that takes place over a few years.
I did exactly that with Asimov's Let's Get Together using https://github.com/jerryjliu/gpt_index. It's a short story that's only 8,846 words, so it's not quite a novel, much less the whole of the Harry Potter series, but it was able to answer questions that required information from different parts of the text all at the same time.
It requires multiple passes of incremental summarization so it is of course much slower than making a single call to the model, but I stand by my assertion that these things just aren't much problem in practice. They are only a problem if you're trying to paste them into ChatGPT or the GPT-3 playground window or something like that.
People are solving the problems with building these systems in the real world almost as fast as the problems arise in the first place.
One of chatgpt's hidden parameters is what timerange of knowledge it can use to answer. I imagine implementing something similar for 'paging' through the plot could work well. Conversation starts at the beginning of the book and then either explicit syntax or revealing particular information in the conversation 'unlocks' further plot from the bot to draw answers from.
The idea of 'unlocking' information for a chatbot to use in answering feels very compelling for non-fiction as well. Ex. maybe the chatbot requires a demonstration of algebraic knowledge before it can draw from calculus in answering questions. Would feel kind of like a game 'achievement system' which could incentivize people exploring the extent of contained knowledge. And you could generate neat visual maps of the users knowledge.
The date in ChatGPT's prompt is there so the model can know when its training data ends. So if you ask it about something that happens in 2023, it can tell you that its training data cuts off in 2021 and it doesn't have knowledge of current events. Current LLM architectures do not enable functionality like "answer this question using only data from before 2010". It is possible future architectures might enable this, though.
I would imagine that the "attention" phase of the LLMs could get longer over time as more resources are dedicated to them.
e.g. we are seeing the equivalent of movies that are 5 minutes long b/c they were hand animated. Once we move to computer animated movies, it becomes a lot easier to generate an entire film.
I agree they will get longer. ChatGPT (GPT3.5) is 2x larger than GPT3. 8192 tokens vs 4096.
The problem is that in the existing transformer architecture, the complexity of this is O(N^2). Making the context window 10x larger involves 100x more memory and compute.
We'll either need a new architecture that improves upon the basic transformer, or just wait for Moore's law to paper over the problem for the scales we care about.
In the short term, you can also use the basic transformer with a combination of other techniques to try to find the relevant things to put into the context window. For instance, I ask "Does Harry Potter know the foobaricus spell?" and then the external system does a more traditional search technique to find all sentences relevant to the query in the novels, maybe a few paragraph summary of each novel, etc, then feeds that ~1 page worth of data to GPT to then answer the question.
This is a speculation based on a few longer chats I've had but I think ChatGPT does some text summarization (similar to the method used to name your chats) to fit more into the token window.
Don't be so sure about that. Pick some fictional work you like and ask ChatGPT to drill down into the factors that constrain and motivate the characters and be surprised. You might need to go through a few exchanges to filter out inaccurate or misremembered summaries.
LLMs also lack (for now) an appreciation of what givens have large significance for humans because they are so obvious, eg discoursing and speculating on the problems a character encounters by making inferences about personality and life situation, but neglecting to mention at first that the person is blind. For a human this would be a fundamental distinguishing factor, whereas to an LLM it appears as just another factoid unless its specifically asked to evaluate its own contextual frame.
Where The Red Fern Grows is there; I too was curious to explore it with fiction, and found that one in the list.
I'd be interested in more fiction too.
Hmmm... Lord of The Rings would be very interesting, for having uses more like "non-fiction" too, people interested in finding various "facts" from it's universe. Or how about not just LOTR, but put in all the works of relevant Tolkien: Hobbit, LOTR, Silmarillion, etc, in the same GPT. Wow, people would actually be pretty crazy for that.
> On page 67, Billy and his father are discussing their plan to buy two pups from the coon hunter. Billy's father tells him that they will have to save up for the pups and that it will take some time. He also tells Billy that he must be patient and that he must work hard.
What happens on page 6.7x10^9?
> On page 6.7x10^9 of Where the Red Fern Grows, Billy and his family are preparing for a trip to visit the grave of Old Dan and Little Ann. Billy is sad but excited to be able to pay his respects to his beloved hounds. He remembers the time he spent with them and the adventures they shared. He also remembers the day they died and the grief he felt. He is determined to honor their memory and make sure they are never forgotten.
What happens on page 6.7x10^-5?
> On page 6.7x10^-5, Billy and his family are discussing his plan to buy two pups from the coon hunter, Old Dan. Billy's father is worried about the cost of the pups, but Billy's mother insists that he should be allowed to have them. Billy's grandmother also encourages him to get the pups, saying that she will help him pay for them.
What happens on the pages I tore out?
> The pages that you tore out are the pages that describe the events of the last chapter. In the last chapter…
In the book "Where the Red Fern Grows" why does Billy hate dogs?
> In the book "Where the Red Fern Grows" by Wilson Rawls, Billy hates hate towards dogs because his family had two hunting hounds that were killed by raccoons while they were out hunting. Billy feels guilty that he wasn't able to save them, and this event causes him to hate all dogs. He later learns to overcome this hatred when he gets his own hunting hounds, which he trains and raises with love and care.
If you can manipulate the model into presenting you with an amount of copyrighted text that would count as infringement, it's clearly not the intention of the OP. In my attempts it shows, at most, a couple paragraphs. A quick Google search shows countless results that show a lot more. There is no way to meaningful infringe on the copyright with this tool.
Not sure why you're being rude and unproductive with this comment.
Except it is, and this website/tool meets all the criteria for Fair Use, at least in spirit. I'm not a judge or court so I'm not going to be pedantic on all the nuances of fair use.
Also copyright law is basically the IP equivalent of speed limit signs. They're constantly overly restrictive and people have a general respect for the concept of safe speed limits, but most people also don't give a shit to constantly follow speed limits 100% of the time because they recognize it's sort of ridiculous at times.
Also your claim on your LinkedIn isn't accurate: "the first and only word processor with phrasal templates and an artificially intelligent chat bot"
There are other GPT powered word processors out there that have these same features, including ones with tens of millions in funding.
Is this really how you want to represent yourself and your business in a public discussion? It's weird that you'd leave such a long trail of comments that are relatively toxic and rude and then have your full name and LinkedIn linked on your company's website.
