I have added two question/answer items to my local FAQ:
Who is the wife of Michael?,“The wife of Michael is Hannah.”
What are the names of the children of Michael?,“The names are George, William and Harry,”
When I ask the following question:
q::who are the sons of Michael?
then I will receive as answer
“The wife of Michael is Hannah.”
whereas the correct answer would have been
"“The names are George, William and Harry,”
But I understand that the system does not know that “sons” and “children” are associated with each other.
I guess I just have to add more training data, like for example
What are the names of the children of Michael?,“The names are George, William and Harry,”
What are the names of the sons of Michael?,“The names are George, William and Harry,”
the model is really bad. i just tried a very simple text on bert large and it gives great results.
text: “this is michael. his wife is hannah. their children are jeremy and anna. he has a brother named james. and a sister named julia.”
who is michael’s daughter? anna
who is michael’s son? jeremy
who is hannah’s daughter? anna
it can handle simple relations
it’s not perfect otherwise
who’s wife is anna? michael
i quickly hacked up a piece of code using huggingface’s transformers (it downloads the checkpoint, which is huge, so be patient)
import torch
from transformers import BertTokenizer, BertForQuestionAnswering
tokenizer = BertTokenizer.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')
model = BertForQuestionAnswering.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')
text=""
question=""
while text != ".":
question=""
text=input("Text?")
while question != ".":
question=input("Question('.' to enter new text)?")
input_text = "[CLS] " + question + " [SEP] " + text + " [SEP]"
input_ids = tokenizer.encode(input_text)
token_type_ids = [0 if i <= input_ids.index(102) else 1 for i in range(len(input_ids))]
s, e = model(torch.tensor([input_ids]), token_type_ids=torch.tensor([token_type_ids]))
all_tokens = tokenizer.convert_ids_to_tokens(input_ids)
print(' '.join(all_tokens[torch.argmax(s) : torch.argmax(e)+1]))
it has general understanding of relationships, a vague notion of who is generally a “brother” or a son or a kid.
try this text:
this is michael. his wife is hannah. their children are jeremy and anna. he has a brother named james. and a sister named julia.
you can ask it differently:
who is his son?
who is his male child?
who is his male kid?
all that produces “jeremy”
while “who is his kids” produces “jeremy and anna”
it has a notion of gender and a lot more.
however, the model is pretty much obsolete. I know it’s not even been a year, but it got replaced with BART and then more recently by T5.
Check out T5 on Natural Questions dataset if you are interested in the topic.
PS This code btw is exactly what various online demos do(including deeppavolov’s), but for console(cli) and probably better than most since it’s a large model.
I leave it here cuz I respect those russian folks that build this website. Hope this will get them some traffic from google. AI research centers are few and far between in russia.