google에서 만든 NAS 패키지 Model Search

언젠가는 다시 볼 것 같아서 간단하게 정리된 것 참고하여 작성

 

 

기존의 어려움들

The challenges for implementing NAS are everywhere. For starters, there are not many NAS frameworks integrated into mainstream deep learning stacks such as TensorFlow or PyTorch. Also, many NAS implementation require a lot of domain expertise to start with a series of architectures that make sense for a given problem. Finally, NAS stacks are really expensive and difficult to use.

 

도메인 지식이 많이 필요했지만, 이것을 극복하기 위해 Google Research에서 Model Search를 제안함.

Model Search는 데이터 사이언티스트들이 최적의 아키텍처를 빠르고 효율적인 방법으로 찾게 해주는 프레임워크

· Run several NAS algorithms for a given dataset.

· Compare different models produced during the search process.

· Create a custom search space for a given problem.

 

---

Model Search 아키텍처는 크게 4가지의 기초 요소를 기반으로 함.

· Model Trainers: These components train and evaluate models asynchronously.

· Search Algorithms: The search algorithm selects the best trained architectures and adds some “mutations” to it and sends it to the trainers for further evaluation.

· Transfer Learning Algorithm: Model Search uses transfer learning techniques such as knowledge distillation to reuse knowledge across different experiments.

· Model Database: The model database persists the results of the experiments in ways that can be reused on different cycles.

 

 

import model_search
from model_search import constants
from model_search import single_trainer
from model_search.data import csv_data

trainer = single_trainer.SingleTrainer(
    data=csv_data.Provider(
        label_index=0,
        logits_dimension=2,
        record_defaults=[0, 0, 0, 0],
        filename="model_search/data/testdata/csv_random_data.csv"),
    spec=constants.DEFAULT_DNN)

trainer.try_models(
    number_models=200,
    train_steps=1000,
    eval_steps=100,
    root_dir="/tmp/run_example",
    batch_size=32,
    experiment_name="example",
    experiment_owner="model_search_user")

 

 

 

 

 

github.com/google/model_search

google/model_search