AutoML library for deep learning

Bug Description

Trying to get started using AutoKeras and finding that most of the example code does not work.

Bug Reproduction

Running the example here: https://autokeras.com/tutorial/structured_data_regression/

Setup Details

Include the details about the versions of:

• OS type and version: MacOS Catalina 10.15.4
• Python: 3.8
• autokeras: master (pulled 20.06.01)
• keras-tuner: 1.0.1
• scikit-learn: 0.23.1
• numpy: 1.18.4
• pandas: 1.0.4
• tensorflow: 2.2.0

Error

ValueError Traceback (most recent call last) in 5 # Evaluate the accuracy of the found model. 6 print('Accuracy: {accuracy}'.format( ----> 7 accuracy=regressor.evaluate(x=test_dataset.drop(columns=['MPG']), y=test_dataset['MPG'])))

~/opt/miniconda2/envs/AutoML/lib/python3.8/site-packages/autokeras-1.0.3-py3.8.egg/autokeras/tasks/structured_data.py in evaluate(self, x, y, batch_size, **kwargs) 133 if isinstance(x, str): 134 x, y = self._read_from_csv(x, y) --> 135 return super().evaluate(x=x, 136 y=y, 137 batch_size=batch_size,

~/opt/miniconda2/envs/AutoML/lib/python3.8/site-packages/autokeras-1.0.3-py3.8.egg/autokeras/auto_model.py in evaluate(self, x, y, **kwargs) 443 """ 444 dataset = self._process_xy(x, y, False) --> 445 return self.tuner.get_best_model().evaluate(x=dataset, **kwargs) 446 447 def export_model(self):

~/opt/miniconda2/envs/AutoML/lib/python3.8/site-packages/autokeras-1.0.3-py3.8.egg/autokeras/engine/tuner.py in get_best_model(self) 43 44 def get_best_model(self): ---> 45 model = super().get_best_models()[0] 46 model.load_weights(self.best_model_path) 47 return model

~/opt/miniconda2/envs/AutoML/lib/python3.8/site-packages/kerastuner/engine/tuner.py in get_best_models(self, num_models) 229 """ 230 # Method only exists in this class for the docstring override. --> 231 return super(Tuner, self).get_best_models(num_models) 232 233 def _deepcopy_callbacks(self, callbacks):

~/opt/miniconda2/envs/AutoML/lib/python3.8/site-packages/kerastuner/engine/base_tuner.py in get_best_models(self, num_models) 236 """ 237 best_trials = self.oracle.get_best_trials(num_models) --> 238 models = [self.load_model(trial) for trial in best_trials] 239 return models 240

~/opt/miniconda2/envs/AutoML/lib/python3.8/site-packages/kerastuner/engine/base_tuner.py in (.0) 236 """ 237 best_trials = self.oracle.get_best_trials(num_models) --> 238 models = [self.load_model(trial) for trial in best_trials] 239 return models 240

~/opt/miniconda2/envs/AutoML/lib/python3.8/site-packages/kerastuner/engine/tuner.py in load_model(self, trial) 154 best_epoch = trial.best_step 155 with hm_module.maybe_distribute(self.distribution_strategy): --> 156 model.load_weights(self._get_checkpoint_fname( 157 trial.trial_id, best_epoch)) 158 return model

~/opt/miniconda2/envs/AutoML/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py in load_weights(self, filepath, by_name, skip_mismatch) 248 raise ValueError('Load weights is not yet supported with TPUStrategy ' 249 'with steps_per_run greater than 1.') --> 250 return super(Model, self).load_weights(filepath, by_name, skip_mismatch) 251 252 def compile(self,

~/opt/miniconda2/envs/AutoML/lib/python3.8/site-packages/tensorflow/python/keras/engine/network.py in load_weights(self, filepath, by_name, skip_mismatch) 1229 else: 1230 try: -> 1231 py_checkpoint_reader.NewCheckpointReader(filepath) 1232 save_format = 'tf' 1233 except errors_impl.DataLossError:

~/opt/miniconda2/envs/AutoML/lib/python3.8/site-packages/tensorflow/python/training/py_checkpoint_reader.py in NewCheckpointReader(filepattern) 93 """ 94 try: ---> 95 return CheckpointReader(compat.as_bytes(filepattern)) 96 # TODO(b/143319754): Remove the RuntimeError casting logic once we resolve the 97 # issue with throwing python exceptions from C++.

ValueError: Unsuccessful TensorSliceReader constructor: Failed to get matching files on ./structured_data_regressor/trial_a2b0718070dcd1d815fe093a8ebb90ab/checkpoints/epoch_52/checkpoint: Not found: ./structured_data_regressor/trial_a2b0718070dcd1d815fe093a8ebb90ab/checkpoints/epoch_52; No such file or directory

Bug Description

When executing pip install autokeras, I get the following message: Could not find a version that satisfies the requirement torch==1.0.1.post2 (from autokeras) (from versions: 0.1.2, 0.1.2.post1) No matching distribution found for torch==1.0.1.post2 (from autokeras)

Reproducing Steps

Steps to reproduce the behavior:

• Step 1: set up anaconda environment
• Step 2: install pytorch via their website's recommended command: conda install pytorch-cpu torchvision-cpu -c pytorch
• Step 3: try to install autokeras via pip install autokeras
• Step 4: get the following output:

Collecting autokeras
  Downloading https://files.pythonhosted.org/packages/c2/32/de74bf6afd09925980340355a05aa6a19e7378ed91dac09e76a487bd136d/autokeras-0.4.0.tar.gz (67kB)
    100% |████████████████████████████████| 71kB 1.3MB/s
Collecting scipy==1.2.0 (from autokeras)
  Downloading https://files.pythonhosted.org/packages/c4/0f/2bdeab43db2b4a75863863bf7eddda8920b031b0a70494fd2665c73c9aec/scipy-1.2.0-cp36-cp36m-win_amd64.whl (31.9MB)
    100% |████████████████████████████████| 31.9MB 508kB/s
Requirement already satisfied: tensorflow==1.13.1 in c:\[...]\lib\site-packages (from autokeras) (1.13.1)
Collecting torch==1.0.1.post2 (from autokeras)
  Could not find a version that satisfies the requirement torch==1.0.1.post2 (from autokeras) (from versions: 0.1.2, 0.1.2.post1)
No matching distribution found for torch==1.0.1.post2 (from autokeras)

Expected Behavior

Autokeras is installed without error.

Setup Details

Include the details about the versions of:

• OS type and version: Windows 10 Version 10.0.17763 Build 17763
• Python: 3.6.8 (anaconda)
• autokeras: 0.4.0
• scikit-learn: 0.20.3
• numpy:1.16.2
• keras: 2.2.4
• scipy:1.2.1
• tensorflow:1.13.1
• pytorch:1.1.0

Additional context

when i run the fit function in mnist example, raised this error: my evironment is windows 10 ,and is strictly requested due to requirements.txt. has someone encountered such a situation? please tell me how to solve
thank you

Traceback (most recent call last): File "", line 1, in File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\search.py", line 231, in search File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main File "C:\Anaconda3\lib\multiprocessing\process.py", line 116, in terminate exitcode = _main(fd) File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 114, in _main self._popen.terminate() AttributeError: 'NoneType' object has no attribute 'terminate' prepare(preparation_data) File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="mp_main") File "C:\Anaconda3\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "C:\Anaconda3\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "C:\Anaconda3\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "D:\autokeras-master\examples\mnist.py", line 9, in clf.fit(x_train,y_train) File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\image\image_supervised.py", line 159, in fit File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\cnn_module.py", line 50, in fit File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\search.py", line 231, in search File "C:\Anaconda3\lib\multiprocessing\process.py", line 116, in terminate self._popen.terminate() AttributeError: 'NoneType' object has no attribute 'terminate'

I have a simple task to find the best CNN architecture for image regression. However, I have a large dataset, which cannot be loaded into memory at one time. It seems in the current release ImageRegressor only supports fit method requiring all the data (x and y) loaded in memory. How can I use generator in Autokeras? I have checked a closed issue #204, but it seems it was not solved.

I have already tried the tf.dataset by converting my generator to tf.dataset, but it didn't work. For example,

    dataset = tf.data.Dataset.from_generator(generate_batch, (tf.float32, tf.float32))
    vq_predictor = ak.ImageRegressor()
    for i, (X, y) in enumerate(dataset):
        X_dataset = tf.data.Dataset.from_tensors(X)
        y_dataset = tf.data.Dataset.from_tensors(y)
        vq_predictor.fit(X_dataset, y_dataset, validation_split=0.2)

Then I got error:

File "C:\Users\junyong\AppData\Local\Continuum\anaconda3\envs\tensorflow2\lib\site-packages\autokeras\tasks\image.py", line 222, in fit **kwargs) File "C:\Users\junyong\AppData\Local\Continuum\anaconda3\envs\tensorflow2\lib\site-packages\autokeras\auto_model.py", line 231, in fit validation_split=validation_split) File "C:\Users\junyong\AppData\Local\Continuum\anaconda3\envs\tensorflow2\lib\site-packages\autokeras\auto_model.py", line 313, in _prepare_data dataset, validation_data = utils.split_dataset(dataset, validation_split) File "C:\Users\junyong\AppData\Local\Continuum\anaconda3\envs\tensorflow2\lib\site-packages\autokeras\utils.py", line 69, in split_dataset raise ValueError('The dataset should at least contain 2 ' ValueError: The dataset should at least contain 2 instances to be split.

Any suggestions are highly appreciated.

Trying to create an image classifier with ~1000 training samples and 7 classes but it throws a runtime error. Is there a way of reducing batch size or something else that can be done to circumvent this?

Following is the error.

RuntimeError: cuda runtime error (2) : out of memory at /pytorch/aten/src/THC/generic/THCStorage.cu:58/usr/lib/python3.5/multiprocessing/semaphore_tracker.py:129: UserWarning: semaphore_tracker: There appear to be 2 leaked semaphores to clean up at shutdown len(cache))

Bug Description

Traceback (most recent call last): File "test.py", line 29, in <module> clf.fit(x_train, y_train, time_limit=60 * 60) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 114, in fit super().fit(x, y, time_limit) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/autokeras/supervised.py", line 129, in fit self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/autokeras/net_module.py", line 65, in fit self.searcher.search(train_data, test_data, int(time_remain)) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/autokeras/search.py", line 200, in search generated_other_info, generated_graph = self.generate(remaining_time, q) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/autokeras/search.py", line 251, in generate remaining_time, multiprocessing_queue) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/autokeras/bayesian.py", line 350, in generate if multiprocessing_queue.qsize() != 0: File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/queues.py", line 117, in qsize return self._maxsize - self._sem._semlock._get_value() NotImplementedError

Reproducing Steps

Just run 'Data with numpy array (.npy) format.' example

Setup Details

Include the details about the versions of:

• OS type and version: macOS 10.14.2
• Python: 3.6

Additional context

seems this is causing the issue: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Queue.qsize

Bug Description

I use autokeras to train a classifier, I can predict or evaluate with the <autokeras.tasks.structured_data.StructuredDataClassifier>. But the model exported by the export_model function can't predict or evaluate. When I try model.evaluate(x_valid,y_valid)it raise the error below, and model.evaluate(x_valid,y_valid) raise the same error. No matter which dataset I use, the iteration ends at 32.

 32/2000 [..............................] - ETA: 48s
---------------------------------------------------------------------------
UnimplementedError                        Traceback (most recent call last)
<ipython-input-20-73f41c3caa9d> in <module>
----> 1 model.evaluate(x_valid,y_valid)

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\keras\engine\training.py in evaluate(self, x, y, batch_size, verbose, sample_weight, steps, callbacks, max_queue_size, workers, use_multiprocessing)
    928         max_queue_size=max_queue_size,
    929         workers=workers,
--> 930         use_multiprocessing=use_multiprocessing)
    931 
    932   def predict(self,

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py in evaluate(self, model, x, y, batch_size, verbose, sample_weight, steps, callbacks, max_queue_size, workers, use_multiprocessing, **kwargs)
    488         sample_weight=sample_weight, steps=steps, callbacks=callbacks,
    489         max_queue_size=max_queue_size, workers=workers,
--> 490         use_multiprocessing=use_multiprocessing, **kwargs)
    491 
    492   def predict(self, model, x, batch_size=None, verbose=0, steps=None,

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py in _model_iteration(self, model, mode, x, y, batch_size, verbose, sample_weight, steps, callbacks, max_queue_size, workers, use_multiprocessing, **kwargs)
    473               mode=mode,
    474               training_context=training_context,
--> 475               total_epochs=1)
    476           cbks.make_logs(model, epoch_logs, result, mode)
    477 

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py in run_one_epoch(model, iterator, execution_function, dataset_size, batch_size, strategy, steps_per_epoch, num_samples, mode, training_context, total_epochs)
    126         step=step, mode=mode, size=current_batch_size) as batch_logs:
    127       try:
--> 128         batch_outs = execution_function(iterator)
    129       except (StopIteration, errors.OutOfRangeError):
    130         # TODO(kaftan): File bug about tf function and errors.OutOfRangeError?

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py in execution_function(input_fn)
     96     # `numpy` translates Tensors to values in Eager mode.
     97     return nest.map_structure(_non_none_constant_value,
---> 98                               distributed_function(input_fn))
     99 
    100   return execution_function

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\eager\def_function.py in __call__(self, *args, **kwds)
    566         xla_context.Exit()
    567     else:
--> 568       result = self._call(*args, **kwds)
    569 
    570     if tracing_count == self._get_tracing_count():

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\eager\def_function.py in _call(self, *args, **kwds)
    636               *args, **kwds)
    637       # If we did not create any variables the trace we have is good enough.
--> 638       return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds)  # pylint: disable=protected-access
    639 
    640     def fn_with_cond(*inner_args, **inner_kwds):

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\eager\function.py in _filtered_call(self, args, kwargs)
   1609          if isinstance(t, (ops.Tensor,
   1610                            resource_variable_ops.BaseResourceVariable))),
-> 1611         self.captured_inputs)
   1612 
   1613   def _call_flat(self, args, captured_inputs, cancellation_manager=None):

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\eager\function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
   1690       # No tape is watching; skip to running the function.
   1691       return self._build_call_outputs(self._inference_function.call(
-> 1692           ctx, args, cancellation_manager=cancellation_manager))
   1693     forward_backward = self._select_forward_and_backward_functions(
   1694         args,

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\eager\function.py in call(self, ctx, args, cancellation_manager)
    543               inputs=args,
    544               attrs=("executor_type", executor_type, "config_proto", config),
--> 545               ctx=ctx)
    546         else:
    547           outputs = execute.execute_with_cancellation(

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\tensorflow_core\python\eager\execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     65     else:
     66       message = e.message
---> 67     six.raise_from(core._status_to_exception(e.code, message), None)
     68   except TypeError as e:
     69     keras_symbolic_tensors = [

c:\users\exia\appdata\local\programs\python\python36\lib\site-packages\six.py in raise_from(value, from_value)

UnimplementedError:  Cast double to string is not supported
	 [[node Cast (defined at <ipython-input-20-73f41c3caa9d>:1) ]] [Op:__inference_distributed_function_1731]

Function call stack:
distributed_function

Reproducing Steps

clf = ak.StructuredDataClassifier(max_trials=10)
clf.fit(x_train, y_train,validation_data=(x_valid, y_valid))
model = clf.export_model()
model.evaluate(x_valid,y_valid)

where x_train is a (5000, 4) numpy.ndarray, y_train is a (5000, 1) numpy.ndarray x_valid is a (2000, 4) numpy.ndarray, y_valid is a (2000, 1) numpy.ndarray no matter I use model.evaluate(x_valid,y_valid) or model.evaluate(x_train,y_train) it raise the same error.

Setup Details

Include the details about the versions of:

• OS type and version: Win10
• Python: 3.6.5
• autokeras: 1.0.1
• scikit-learn:0.22
• numpy:1.18.0
• scipy:1.4.1
• tensorflow:2.1.0

Today objective = "val_f1" returns an error Failed to train : <class 'ValueError'> : Could not infer optimization direction ("min" or "max") for unknown metric "val_f1". Please specify the objective asa kerastuner.Objective, for example kerastuner.Objective("val_f1", direction="min").

import pandas as pd
import numpy as np
import autokeras as ak
from tensorflow.keras.datasets import cifar10
from tensorflow.python.keras.utils.data_utils import Sequence
from tensorflow.keras.models import model_from_json
import os
def build_model():
    input_layer =ak.Input()
    cnn_layer = ak.ConvBlock()(input_layer)
    cnn_layer2 =ak.ConvBlock()(cnn_layer)
    dense_layer =ak.DenseBlock()(cnn_layer2)
    dense_layer2 =ak.DenseBlock()(dense_layer)
    output_layer =ak.ClassificationHead(num_classes=10)(dense_layer2)
    automodel =ak.auto_model.AutoModel(input_layer,output_layer,max_trials=20,seed=123,project_name="automl")
    return automodel

def build():
    ((trainX,trainY),(testX,testY))=cifar10.load_data()
    automodel = build_model()
    automodel.fit(trainX,trainY,validation_split=0.2,epochs=40,batch_size=64)#error here

if __name__ == '__main__':
    build()

i got this error even trying the example in the docs

    automodel.fit(trainX,trainY,validation_split=0.2,epochs=40,batch_size=64)
  File "S:\Anaconda\envs\tensor37\lib\site-packages\autokeras\auto_model.py", line 276, in fit
    validation_split=validation_split,
  File "S:\Anaconda\envs\tensor37\lib\site-packages\autokeras\auto_model.py", line 409, in _prepare_data
    dataset, validation_split
  File "S:\Anaconda\envs\tensor37\lib\site-packages\autokeras\utils\data_utils.py", line 47, in split_dataset
    "The dataset should at least contain 2 batches to be split."
ValueError: The dataset should at least contain 2 batches to be split.

autokeras 1.0.8 keras 2.3.1 tensorflow 2.1.0 numpy 1.19.1 pandas 1.1.1 keras-tuner 1.0.2rc1 python 3.7.7

Bug Description

I can't save the best model as h5 or even a tf file

Bug Reproduction

Code for reproducing the bug:

model = regressor.export_model()

model.save('testmodel.h5')

gives me

NotImplementedError: Save or restore weights that is not an instance of `tf.Variable` is not supported in h5, use `save_format='tf'` instead. Got a model or layer CategoricalEncoding with weights ....

It suggests I use tf instead of h5, but when I do that I get this error

model.save('testmodel', save_format='tf')

2020-04-03 14:44:31.041453: W tensorflow/python/util/util.cc:329] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
*** RuntimeError: Attempting to capture an EagerTensor without building a function.

Data used by the code:

Expected Behavior

Save the best model in either h5 or tf format

Setup Details

• OS type and version: Ubuntu 18.04.01
• Python: 3.6.6
• autokeras: 1.0.2
• scikit-learn: 0.20.3
• numpy: 1.18.2
• pandas: 1.0.1
• tensorflow: 2.2.0-dev20200401 (nightly)

Additional context

I also can't load the model json after saving it but I think that's related to #1023

After training the model using AutoKeras, can we somehow export the structure of the model to be recreated in native TensorFlow and Keras (no dependencies on Autokeras lib)?

Can this process be automated, or can we just print out all the layers in the trained model and recreate that in TensorFlow/Keras?

I think this is a pretty vital feature that is useful in a lot of use-cases. It also helps in re-creating the best-found model in Pytorch and customize to our needs.

Bug Description

After installing AutoKeras and depencies according to instructions, trying to import autokeras throws the exception: cannot import name 'keras' from 'tensorflow' (unknown location)

Bug Reproduction

from autokeras import StructuredDataClassifier

Expected Behavior

Import succeeds without error.

Setup Details

Include the details about the versions of:

• Windows 10
• Python: 3.8.0
• autokeras: 1.0.20
• keras-tuner: 1.2.0.dev0
• keras: 2.11.0
• scikit-learn: 1.2.0
• numpy: 1.22.4
• pandas: 1.5.2
• tensorflow: 2.11.0

Additional context

The exception is thrown in many instances.

Example: This tutorial website:

import pandas as pd
import tensorflow as tf
import autokeras as ak  <--- same exception thrown

Bug Description

Bug Reproduction

Code for reproducing the bug: Installation:

conda create --name env python=3.9.13
conda activate env
conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0
pip install tensorflow
conda install jupyter
pip install autokeras
pip install tqdm
pip install scikit-learn
pip install tensorflow-datasets

Code:

import os
import sys

sys.path.append("/home/julia/dir/src")
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

import logging
import contextlib

from tqdm import tqdm
import tensorflow_datasets as tfds
from tensorflow.compat.v1 import ConfigProto
from tensorflow.compat.v1 import InteractiveSession

from automl.vanilla_autokeras import train_and_test_autokeras

if __name__ == "__main__":
    os.system("clear")
    SVM_DATA_PATH="/data/julia/data/svm_data"
    RESULTS_PATH = "/data/julia/results"
    EXPERIMENT = "autokeras"

    config = ConfigProto()
    config.gpu_options.allow_growth = True
    session = InteractiveSession(config=config)

    logging.basicConfig(
        filename=f"/data/julia/logs/{EXPERIMENT}.log", level=logging.WARNING
    )
    logging.captureWarnings(True)

    with open(os.path.join(RESULTS_PATH, EXPERIMENT + ".csv"), "w") as f:
        f.write("index,acc,precision,recall,f1\n")

    pbar = tqdm(range(10), leave=True, position=0)
    for i in pbar:
        print("#############")
        print("# Autokeras #")
        print("#############")
        
        print("Get data")
        train=tfds.as_numpy(tfds.load("data", split="train", data_dir="/data/julia/data/tfds",as_supervised=False,batch_size=-1))
        test=tfds.as_numpy(tfds.load("data", split="test", data_dir="/data/julia/data/tfds",as_supervised=False,batch_size=-1))
        val=tfds.as_numpy(tfds.load("data", split="val", data_dir="/data/julia/data/tfds",as_supervised=False,batch_size=-1))

        print("Train model")
        acc,precision,recall,f1 = train_and_test_autokeras(train["image"],test["image"],val["image"],train["label"],test["label"],val["label"],f"autokeras_{i}")

        with open(os.path.join(RESULTS_PATH, EXPERIMENT + ".csv"), "a") as f:
            f.write(
                ",".join(
                    str(res) for res in [i, acc, precision, recall, f1]
                )
                + "\n"
            )

        if i != 9:
            os.system("clear")

With the ImageClassifier training/testing code:

clf =ak.ImageClassifier(overwrite=True,project_name=name,directory="/data/julia/models/")
clf.fit(X_train,y_train,validation_data=(X_val,y_val))
predicted_y = clf.predict(X_test)

# get acc, precision, recall, f1
acc = accuracy_score(y_test, predicted_y)
precision = precision_score(y_test, predicted_y,average='macro')
recall = recall_score(y_test, predicted_y,average='macro')
f1 = f1_score(y_test, predicted_y,average='macro')

Expected Behavior

Setup Details

Include the details about the versions of:

• OS type and version: Debian 5.10.149-2
• Python: 3.9.13
• autokeras: 1.0.20
• keras-tuner:1.1.3
• scikit-learn:1.2.0
• numpy:1.19.5
• pandas:1.4.4
• tensorflow: 2.11

Bug Description

AutoModel constructor does not scale well to 5000+ inputs & outputs.

Bug Reproduction

Code for reproducing the bug:

X_train = []  # of size 5000
Y_train = []  # of size 5000

am: AutoModel = ak.AutoModel(inputs=[ak.Input() for _ in range(0, len(X_train))],
                                 outputs=[ak.RegressionHead() for _ in range(0, len(Y_train))])

Data used by the code:

Expected Behavior

Setup Details

Include the details about the versions of:

• OS type and version:
• Python: 3.10.6
• autokeras: 1.02
• keras-tuner:
• scikit-learn:
• numpy: 1.23.3
• pandas:
• tensorflow:

Additional context

Using a server grade CPU.

Feature Description

Hi!

Would it be possible to have TabNet support for structured data? Info about TabNet: https://arxiv.org/abs/1908.07442

It seems to have good performance for structured data.

Thanks!

I'm testing AutoKeras on a recommender system. Usually recommenders are developed with a fixed training set. When a good result is found, the architecture is finalized. When deployed the model will be trained continuously with all users' online actions, so the development process is very different from the development processes for other models, e.g. image classification. I haven't found any AutoKeras API to support this kind of training, is it supported ?