How to train DeepPavlov FAQ iteratively

michaelwechner · January 16, 2020, 7:59am

Hi

I am using

python3 -m deeppavlov train tfidf_logreg_en_faq

whereas I in the dataset_reader I have configured

“data_path”: “/home/foobar/faq_school_en.csv”

Instead updating the CSV and re-train DeepPavlov I would like to append new QUESTION:ANSWER tupples iteratively.

Is this possible somehow?

The most elegant would be if one could do this via the riseapi
python3 -m deeppavlov riseapi tfidf_logreg_en_faq
http://0.0.0.0:5000/train?question=…&answer=…

Thanks

Michael

dilyararimovna · January 17, 2020, 12:26pm

Hello!

Although Logistic Regression has warm_start parameter and could be retrained starting from custom initialization, it does not imply warm start with changing number of classes. Whereas when you add new question-answer pair, you add one more class. So, it’s not possible to iteratively add classes to the model now.
Anyway, logreg is fitted very fast, so, just add your own script to re-train model any time you add more question-answer pairs.

michaelwechner · January 17, 2020, 12:49pm

Hi @dilyararimovna, thanks very much for your reply!

michaelwechner · February 9, 2020, 11:07am

I have created a REST interface to retrain and rebuild the model, whereas please see

github.com

wyona/DeepPavlov/blob/riseapi_retrain/deeppavlov/utils/server/server.py

# Copyright 2017 Neural Networks and Deep Learning lab, MIPT
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import asyncio
import argparse
from deeppavlov.core.common.file import find_config
from deeppavlov.core.commands.train import train_evaluate_model_from_config
from collections import namedtuple
from logging import getLogger

This file has been truncated. show original

In think this works well for a “small” set of questions/answers.

Topic		Replies	Views
Training DeepPavlov without restarting riseapi Documentation	1	423	February 9, 2020
Fitting of loaded faq model can not be continued Models	3	1027	March 11, 2020
Retrain My Custom Train Model Models	5	383	August 26, 2021
DeepPavlov as a multi-tenant service DeepPavlov Library	3	381	September 22, 2020
How to change dataset for the demo Simple intent recognition question answering bot? DeepPavlov Library	2	329	April 13, 2022

How to train DeepPavlov FAQ iteratively

Related topics