• PyTorch-Forecasting version: 0.8.5
• PyTorch version: 1.8.1
• Python version: 3.8.10
• Operating System: linux 3.10.0-1160.25.1.el7.x86_64

according to doc: https://pytorch-forecasting.readthedocs.io/en/latest/api/pytorch_forecasting.data.timeseries.TimeSeriesDataSet.html the target parameter can be set to a list of strings indicating multiple variables for prediction However, during run time, the code spits the following error:

TypeError: new() received an invalid combination of arguments - got (list, int), but expected one of:
 * (*, torch.device device)
      didn't match because some of the arguments have invalid types: (list, int)
 * (torch.Storage storage)
 * (Tensor other)
 * (tuple of ints size, *, torch.device device)
 * (object data, *, torch.device device)

I'm trying to forecast a time series. For that, I decided to redo the tutorial "Demand forecasting with the Temporal Fusion Transformer". As the version that is on the GitHub seems to be more recent, I solved the instructions that are there. However, when executing the instruction trainer.fit(tft, train_dataloaders = train_dataloader, val_dataloaders = val_dataloader,), the following error message appears: "dictionary update sequence element #0 has length 1; 2 is required".

I've tried all the possibilities I knew, but the error persists.

My notebook on Google Colab: https://colab.research.google.com/drive/1NX-ah_Nuqyt6m2Wsn2WcKkadRariISFu?usp=sharing

• PyTorch-Forecasting version: 0.8.3
• PyTorch version: 1.7.1
• Python version: 3.9
• Operating System: win10

Expected behavior

I execute the Temporal Fusion Transformer for power bus load forecasting. The resolution of the data is 15 minutes, which means that there are 96 data points in a day. I want to enter the data from the previous day to predict the 96 data values of the day.

Actual behavior

But the predicted curve is a straight line. The predicted result is 96 nearly the same value.

My model (DeepAR) is performing way better than I think it should.

I've done some hunting and it appears that even in test mode input_vector at this point contains scaled target values.

I tried following things further to here, and note that there is specific handling in place for when n_samples is not None (which is the case when using DeepAR.predict()), but I can't see where the scaled values masked / excluded.

Am I just lost / paranoid or is there leakage somewhere?

Hello, im stallion.py but im getting few problems: using gpu = 1 in the trainer return

models/temporal_fusion_transformer/init.py", line 793, in _log_interpretation dim=0

using only the cpu it works but return this error:

AttributeError: module 'tensorflow._api.v1.io.gfile' has no attribute 'get_filesystem' which apparently it is due to pytorch and tensorboard incompatibility but i managed to install pytorch-forecasting without problems so could you tell me what version of pythorch/tensorboard are you using?

thanks

• PyTorch-Forecasting version: 0.8.3
• PyTorch version: 1.7.1
• Python version: 3.7.9
• Operating System: Linux

Expected behaviour

I am training a simple time series forecasting on temperature prediction problem. I have replicated the Stallion code for my dataset. I get an index out of bounds error as soon as training starts. I am unable to debug it.

Code to reproduce the problem

### Data Set and Data Loader
max_prediction_length = 6
max_encoder_length = 24
training_cutoff = train_data["time_idx"].max() - max_prediction_length

training = TimeSeriesDataSet(
    train_data[lambda x: x.time_idx <= training_cutoff],
    time_idx="time_idx",
    target="meantemp",
    group_ids= ["group"],
    min_encoder_length=max_encoder_length // 2,  # keep encoder length long (as it is in the validation set)
    max_encoder_length=max_encoder_length,
    min_prediction_length=1,
    max_prediction_length=max_prediction_length,
    time_varying_known_categoricals=["day", "month"],
    time_varying_known_reals=["time_idx", "humidity","wind_speed"],
    time_varying_unknown_reals=['meantemp'],
#     target_normalizer=GroupNormalizer(
#         groups=["group"], transformation="softplus"
#     ),  # use softplus and normalize by group
    
    target_normalizer= EncoderNormalizer(transformation='softplus'),  # use softplus and normalize by group
    add_relative_time_idx=True,
    allow_missings=True,
    add_target_scales=True,
    add_encoder_length=True,
)


validation = TimeSeriesDataSet.from_dataset(training, train_data, predict=True, 
                                            min_prediction_idx=training_cutoff + 1,
                                            stop_randomization=False)

# create dataloaders for model
batch_size = 32  # set this between 32 to 128
train_dataloader = training.to_dataloader(train=True, batch_size=batch_size, num_workers=0)
val_dataloader = validation.to_dataloader(train=False, batch_size=batch_size, num_workers=4)

### Training code
early_stop_callback = EarlyStopping(monitor="val_loss", min_delta=1e-4, patience=10, verbose=False, mode="min")
lr_logger = LearningRateMonitor()  # log the learning rate
logger = TensorBoardLogger("lightning_logs")  # logging results to a tensorboard

trainer = pl.Trainer(
    max_epochs=30,
    gpus=0,
    weights_summary="top",
    gradient_clip_val=0.1,
    limit_train_batches=30,  # coment in for training, running valiation every 30 batches
    # fast_dev_run=True,  # comment in to check that networkor dataset has no serious bugs
    callbacks=[lr_logger, early_stop_callback],
    logger=logger,
)


tft = TemporalFusionTransformer.from_dataset(
    training,
    learning_rate=0.1,
    hidden_size=16,
    attention_head_size=1,
    dropout=0.1,
    hidden_continuous_size=8,
    output_size=7,  # 7 quantiles by default
    loss=QuantileLoss(),
    log_interval=10,  # uncomment for learning rate finder and otherwise, e.g. to 10 for logging every 10 batches
    reduce_on_plateau_patience=4,
)
print(f"Number of parameters in network: {tft.size()/1e3:.1f}k")

# fit network
trainer.fit(
    tft,
    train_dataloader=train_dataloader,
    val_dataloaders=val_dataloader,
)

The error comes on the fit function. See below:

RuntimeError Traceback (most recent call last) in 3 tft, 4 train_dataloader=train_dataloader, ----> 5 val_dataloaders=val_dataloader, 6 )

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py in fit(self, model, train_dataloader, val_dataloaders, datamodule) 508 self.call_hook('on_fit_start') 509 --> 510 results = self.accelerator_backend.train() 511 self.accelerator_backend.teardown() 512

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py in train(self) 55 def train(self): 56 self.trainer.setup_trainer(self.trainer.model) ---> 57 return self.train_or_test() 58 59 def teardown(self):

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py in train_or_test(self) 72 else: 73 self.trainer.train_loop.setup_training() ---> 74 results = self.trainer.train() 75 return results 76

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py in train(self) 559 with self.profiler.profile("run_training_epoch"): 560 # run train epoch --> 561 self.train_loop.run_training_epoch() 562 563 if self.max_steps and self.max_steps <= self.global_step:

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py in run_training_epoch(self) 548 # ------------------------------------ 549 with self.trainer.profiler.profile("run_training_batch"): --> 550 batch_output = self.run_training_batch(batch, batch_idx, dataloader_idx) 551 552 # when returning -1 from train_step, we end epoch early

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py in run_training_batch(self, batch, batch_idx, dataloader_idx) 716 717 # optimizer step --> 718 self.optimizer_step(optimizer, opt_idx, batch_idx, train_step_and_backward_closure) 719 720 else:

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py in optimizer_step(self, optimizer, opt_idx, batch_idx, train_step_and_backward_closure) 491 on_tpu=self.trainer.use_tpu and TPU_AVAILABLE, 492 using_native_amp=using_native_amp, --> 493 using_lbfgs=is_lbfgs, 494 ) 495

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/core/lightning.py in optimizer_step(self, epoch, batch_idx, optimizer, optimizer_idx, optimizer_closure, on_tpu, using_native_amp, using_lbfgs) 1296 1297 """ -> 1298 optimizer.step(closure=optimizer_closure) 1299 1300 def optimizer_zero_grad(

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/core/optimizer.py in step(self, closure, make_optimizer_step, *args, **kwargs) 284 285 if make_optimizer_step: --> 286 self.__optimizer_step(*args, closure=closure, profiler_name=profiler_name, **kwargs) 287 else: 288 # make sure to call optimizer_closure when accumulating

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/core/optimizer.py in __optimizer_step(self, closure, profiler_name, *args, **kwargs) 142 else: 143 with trainer.profiler.profile(profiler_name): --> 144 optimizer.step(closure=closure, *args, **kwargs) 145 146 accelerator_backend = trainer.accelerator_backend

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_forecasting/optim.py in step(self, closure) 129 closure: A closure that reevaluates the model and returns the loss. 130 """ --> 131 _ = closure() 132 loss = None 133 # note - below is commented out b/c I have other work that passes back

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py in train_step_and_backward_closure() 711 opt_idx, 712 optimizer, --> 713 self.trainer.hiddens 714 ) 715 return None if result is None else result.loss

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py in training_step_and_backward(self, split_batch, batch_idx, opt_idx, optimizer, hiddens) 804 with self.trainer.profiler.profile("training_step_and_backward"): 805 # lightning module hook --> 806 result = self.training_step(split_batch, batch_idx, opt_idx, hiddens) 807 self._curr_step_result = result 808

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py in training_step(self, split_batch, batch_idx, opt_idx, hiddens) 317 model_ref._current_fx_name = 'training_step' 318 model_ref._results = Result() --> 319 training_step_output = self.trainer.accelerator_backend.training_step(args) 320 self.trainer.logger_connector.cache_logged_metrics() 321

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/accelerators/cpu_accelerator.py in training_step(self, args) 60 61 def training_step(self, args): ---> 62 return self._step(self.trainer.model.training_step, args) 63 64 def validation_step(self, args):

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_lightning/accelerators/cpu_accelerator.py in _step(self, model_step, args) 56 output = model_step(*args) 57 else: ---> 58 output = model_step(*args) 59 return output 60

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_forecasting/models/base_model.py in training_step(self, batch, batch_idx) 263 """ 264 x, y = batch --> 265 log, _ = self.step(x, y, batch_idx) 266 # log loss 267 self.log("train_loss", log["loss"], on_step=True, on_epoch=True, prog_bar=True)

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_forecasting/models/temporal_fusion_transformer/init.py in step(self, x, y, batch_idx) 545 """ 546 # extract data and run model --> 547 log, out = super().step(x, y, batch_idx) 548 # calculate interpretations etc for latter logging 549 if self.log_interval > 0:

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_forecasting/models/base_model.py in step(self, x, y, batch_idx, **kwargs) 378 self.log_metrics(x, y, out) 379 if self.log_interval > 0: --> 380 self.log_prediction(x, out, batch_idx) 381 log = {"loss": loss, "n_samples": x["decoder_lengths"].size(0)} 382

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_forecasting/models/base_model.py in log_prediction(self, x, out, batch_idx) 487 log_indices = [0] 488 for idx in log_indices: --> 489 fig = self.plot_prediction(x, out, idx=idx, add_loss_to_title=True) 490 tag = f"{['Val', 'Train'][self.training]} prediction" 491 if self.training:

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_forecasting/models/temporal_fusion_transformer/init.py in plot_prediction(self, x, out, idx, plot_attention, add_loss_to_title, show_future_observed, ax) 708 # add attention on secondary axis 709 if plot_attention: --> 710 interpretation = self.interpret_output(out) 711 for f in to_list(fig): 712 ax = f.axes[0]

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_forecasting/models/temporal_fusion_transformer/init.py in interpret_output(self, out, reduction, attention_prediction_horizon, attention_as_autocorrelation) 596 597 # histogram of decode and encode lengths --> 598 encoder_length_histogram = integer_histogram(out["encoder_lengths"], min=0, max=self.hparams.max_encoder_length) 599 decoder_length_histogram = integer_histogram( 600 out["decoder_lengths"], min=1, max=out["decoder_variables"].size(1)

~/miniconda3/envs/torch/lib/python3.7/site-packages/pytorch_forecasting/utils.py in integer_histogram(data, min, max) 32 max = uniques.max() 33 hist = torch.zeros(max - min + 1, dtype=torch.long, device=data.device).scatter( ---> 34 dim=0, index=uniques - min, src=counts 35 ) 36 return hist

RuntimeError: index 25 is out of bounds for dimension 0 with size 25

Hi @jdb78,

After getting the TFT model working well for one group of data based on our last convo (and updating to the latest version of the library), I'm getting an odd tensor dimension error when I try to train my model across multiple groups on my data. Specifically on epoch 15 I get this error/trace:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-32-14fda4f79b4a> in <module>
      1 # Train model
----> 2 trainer.fit(
      3     tft,
      4     train_dataloader = train_dataloader,
      5     val_dataloaders = val_dataloader

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_lightning/trainer/states.py in wrapped_fn(self, *args, **kwargs)
     46             if entering is not None:
     47                 self.state = entering
---> 48             result = fn(self, *args, **kwargs)
     49 
     50             # The INTERRUPTED state can be set inside the run function. To indicate that run was interrupted

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py in fit(self, model, train_dataloader, val_dataloaders, datamodule)
   1071             self.accelerator_backend = GPUBackend(self)
   1072             model = self.accelerator_backend.setup(model)
-> 1073             results = self.accelerator_backend.train(model)
   1074 
   1075         elif self.use_tpu:

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_lightning/accelerators/gpu_backend.py in train(self, model)
     49 
     50     def train(self, model):
---> 51         results = self.trainer.run_pretrain_routine(model)
     52         return results
     53 

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py in run_pretrain_routine(self, model)
   1237 
   1238         # CORE TRAINING LOOP
-> 1239         self.train()
   1240 
   1241     def _run_sanity_check(self, ref_model, model):

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py in train(self)
    392                 # RUN TNG EPOCH
    393                 # -----------------
--> 394                 self.run_training_epoch()
    395 
    396                 if self.max_steps and self.max_steps <= self.global_step:

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py in run_training_epoch(self)
    548 
    549         # process epoch outputs
--> 550         self.run_training_epoch_end(epoch_output, checkpoint_accumulator, early_stopping_accumulator, num_optimizers)
    551 
    552         # checkpoint callback

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py in run_training_epoch_end(self, epoch_output, checkpoint_accumulator, early_stopping_accumulator, num_optimizers)
    662             # run training_epoch_end
    663             # a list with a result per optimizer index
--> 664             epoch_output = model.training_epoch_end(epoch_output)
    665 
    666             if isinstance(epoch_output, Result):

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_forecasting/models/base_model.py in training_epoch_end(self, outputs)
    133 
    134     def training_epoch_end(self, outputs):
--> 135         log, _ = self.epoch_end(outputs, label="train")
    136         return log
    137 

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_forecasting/models/temporal_fusion_transformer/__init__.py in epoch_end(self, outputs, label)
    613         log, out = super().epoch_end(outputs, label=label)
    614         if self.log_interval(label == "train") > 0:
--> 615             self._log_interpretation(out, label=label)
    616         return log, out
    617 

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_forecasting/models/temporal_fusion_transformer/__init__.py in _log_interpretation(self, outputs, label)
    820         """
    821         # extract interpretations
--> 822         interpretation = {
    823             name: torch.stack([x["interpretation"][name] for x in outputs]).sum(0)
    824             for name in outputs[0]["interpretation"].keys()

~/anaconda3/envs/forecasting/lib/python3.8/site-packages/pytorch_forecasting/models/temporal_fusion_transformer/__init__.py in <dictcomp>(.0)
    821         # extract interpretations
    822         interpretation = {
--> 823             name: torch.stack([x["interpretation"][name] for x in outputs]).sum(0)
    824             for name in outputs[0]["interpretation"].keys()
    825         }

RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 6 and 7 in dimension 1 at /tmp/pip-req-build-8yht7tdu/aten/src/THC/generic/THCTensorMath.cu:71

At first I thought this could be because not every state has the same amount of data available. For example, for select states, the number of days for which data exist are:

state
CA    218
FL    218
GA    218
NY    218
TX    218
WA    260

So I tried just getting the last 218 observations for each state and resetting time_idx, but the same tensor error was raised.

Right now I'm only including the state as a group variable and am doing univariate time series modeling, so no other data are in the TFT.

Also, the learning rate finder works, this is just the training that's failing. Really odd.

I've made my code available here: https://drive.google.com/file/d/1r1w2tHZJrr8iXVqw7U5_qL1x4iOUsauk/view?usp=sharing. The data I'm playing with are on COVID, and the notebook includes a pd.read_csv() call that read in all the data for you from an online source, so you should be able to run it on your own without any issues.

Any thoughts? Again, I greatly appreciate your feedback as I'm new to deep learning on time series data.

Thanks in advance!

Best, Alex

• PyTorch-Forecasting version: 0.9.0
• PyTorch version: 1.9.0
• Python version: 1.3.8
• Operating System: Debian 9 Linux 4.14

Expected behavior

I created a new virtual env and use pip install pytorch-forecasting and pip install pyarrow to run the tutorial code of TFT model.

Actual behavior

I got the following error:

trainer.fit(...
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~/qps_old/test.py in 
      165     tft,
      166     train_dataloader=train_dataloader,
----> 167     val_dataloaders=val_dataloader,
      168 )
      169 

~/anaconda3/envs/envtest/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py in fit(self, model, train_dataloader, val_dataloaders, datamodule)
    458         )
    459 
--> 460         self._run(model)
    461 
    462         assert self.state.stopped

~/anaconda3/envs/envtest/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py in _run(self, model)
    753 
    754         # plugin will setup fitting (e.g. ddp will launch child processes)
--> 755         self.pre_dispatch()
    756 
    757         # dispatch `start_training` or `start_evaluating` or `start_predicting`

~/anaconda3/envs/envtest/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py in pre_dispatch(self)
    785             self.logger.log_hyperparams(self.lightning_module.hparams_initial)
    786             self.logger.log_graph(self.lightning_module)
--> 787             self.logger.save()
    788 
    789     def post_dispatch(self):

~/anaconda3/envs/envtest/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py in wrapped_fn(*args, **kwargs)
     47     def wrapped_fn(*args, **kwargs):
     48         if rank_zero_only.rank == 0:
---> 49             return fn(*args, **kwargs)
     50 
     51     return wrapped_fn

~/anaconda3/envs/envtest/lib/python3.7/site-packages/pytorch_lightning/loggers/tensorboard.py in save(self)
    236         # save the metatags file if it doesn't exist and the log directory exists
    237         if self._fs.isdir(dir_path) and not self._fs.isfile(hparams_file):
--> 238             save_hparams_to_yaml(hparams_file, self.hparams)
    239 
    240     @rank_zero_only

~/anaconda3/envs/envtest/lib/python3.7/site-packages/pytorch_lightning/core/saving.py in save_hparams_to_yaml(config_yaml, hparams)
    398     for k, v in hparams.items():
    399         try:
--> 400             yaml.dump(v)
    401         except TypeError:
    402             warn(f"Skipping '{k}' parameter because it is not possible to safely dump to YAML.")

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/__init__.py in dump(data, stream, Dumper, **kwds)
    288     If stream is None, return the produced string instead.
    289     """
--> 290     return dump_all([data], stream, Dumper=Dumper, **kwds)
    291 
    292 def safe_dump_all(documents, stream=None, **kwds):

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/__init__.py in dump_all(documents, stream, Dumper, default_style, default_flow_style, canonical, indent, width, allow_unicode, line_break, encoding, explicit_start, explicit_end, version, tags, sort_keys)
    276         dumper.open()
    277         for data in documents:
--> 278             dumper.represent(data)
    279         dumper.close()
    280     finally:

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent(self, data)
     25 
     26     def represent(self, data):
---> 27         node = self.represent_data(data)
     28         self.serialize(node)
     29         self.represented_objects = {}

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_data(self, data)
     50             for data_type in data_types:
     51                 if data_type in self.yaml_multi_representers:
---> 52                     node = self.yaml_multi_representers[data_type](self, data)
     53                     break
     54             else:

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_object(self, data)
    341                 and isinstance(state, dict) and newobj:
    342             return self.represent_mapping(
--> 343                     'tag:yaml.org,2002:python/object:'+function_name, state)
    344         if not listitems and not dictitems  \
    345                 and isinstance(state, dict) and not state:

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_mapping(self, tag, mapping, flow_style)
    116         for item_key, item_value in mapping:
    117             node_key = self.represent_data(item_key)
--> 118             node_value = self.represent_data(item_value)
    119             if not (isinstance(node_key, ScalarNode) and not node_key.style):
    120                 best_style = False

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_data(self, data)
     50             for data_type in data_types:
     51                 if data_type in self.yaml_multi_representers:
---> 52                     node = self.yaml_multi_representers[data_type](self, data)
     53                     break
     54             else:

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_object(self, data)
    341                 and isinstance(state, dict) and newobj:
    342             return self.represent_mapping(
--> 343                     'tag:yaml.org,2002:python/object:'+function_name, state)
    344         if not listitems and not dictitems  \
    345                 and isinstance(state, dict) and not state:

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_mapping(self, tag, mapping, flow_style)
    116         for item_key, item_value in mapping:
    117             node_key = self.represent_data(item_key)
--> 118             node_value = self.represent_data(item_value)
    119             if not (isinstance(node_key, ScalarNode) and not node_key.style):
    120                 best_style = False

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_data(self, data)
     50             for data_type in data_types:
     51                 if data_type in self.yaml_multi_representers:
---> 52                     node = self.yaml_multi_representers[data_type](self, data)
     53                     break
     54             else:

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_object(self, data)
    344         if not listitems and not dictitems  \
    345                 and isinstance(state, dict) and not state:
--> 346             return self.represent_sequence(tag+function_name, args)
    347         value = {}
    348         if args:

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_sequence(self, tag, sequence, flow_style)
     90         best_style = True
     91         for item in sequence:
---> 92             node_item = self.represent_data(item)
     93             if not (isinstance(node_item, ScalarNode) and not node_item.style):
     94                 best_style = False

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_data(self, data)
     46         data_types = type(data).__mro__
     47         if data_types[0] in self.yaml_representers:
---> 48             node = self.yaml_representers[data_types[0]](self, data)
     49         else:
     50             for data_type in data_types:

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_tuple(self, data)
    284 
    285     def represent_tuple(self, data):
--> 286         return self.represent_sequence('tag:yaml.org,2002:python/tuple', data)
    287 
    288     def represent_name(self, data):

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_sequence(self, tag, sequence, flow_style)
     90         best_style = True
     91         for item in sequence:
---> 92             node_item = self.represent_data(item)
     93             if not (isinstance(node_item, ScalarNode) and not node_item.style):
     94                 best_style = False

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_data(self, data)
     50             for data_type in data_types:
     51                 if data_type in self.yaml_multi_representers:
---> 52                     node = self.yaml_multi_representers[data_type](self, data)
     53                     break
     54             else:

~/anaconda3/envs/envtest/lib/python3.7/site-packages/yaml/representer.py in represent_object(self, data)
    329         if dictitems is not None:
    330             dictitems = dict(dictitems)
--> 331         if function.__name__ == '__newobj__':
    332             function = args[0]
    333             args = args[1:]

AttributeError: 'functools.partial' object has no attribute '__name__'

This is really strange, I thought it might be caused by version compatibility of pytorch-lightning and pytorch-forecasting (or tensorboard and PyYAML). I tried to use different version of pytorch-forecating, pytorch-lightning, but no progress.

Code to reproduce the problem

Just install the pytorch-forecasting and follow the turorial.

pip install pytorch-forecasting
pip install pyarrow

#os.chdir("../../..")   #only change in the tutorial code to avoid permission error.

Thank you!

Solution

Found the problem (although I don't know why). The problem lies in pandas version. I tested and found that pandas==1.2.x works fine but pandas==1.3.x will raise the error. Now pip install pytorch-forecasting and pip install --upgrade pandas==1.2.5 is OK to run.

Hi, Really appreciate your work on the TFT.

I am trying to use my own dataset in the code but there seems to be a bug due to which the dataset is not being loaded properly for validation. The train dataloader is good. but the validation dataloader only has one batch and also validation(TimeSeriesDataSet) has only 1 entry.

Below is my complete code

data = load_csv()
data['date']= pd.to_datetime(data['date'])
data.reset_index(inplace=True, drop=True)
data.reset_index(inplace=True)
data.rename(columns={'index':'time_idx'}, inplace=True) # I use index as time_idx since my data is of minute frequency

validation_len = int(len(data) * 0.1)
training_cutoff = int(len(data)) - validation_len

max_encode_length = 36
max_prediction_length = 6

print('Len of training data is : ',len(data[:training_cutoff]))
print('Len of val data is : ',len(data[training_cutoff:]))
training = TimeSeriesDataSet(
    data[:training_cutoff],
    time_idx="time_idx",
    target="T",
    group_ids=["Symbol"],
    max_encoder_length=max_encode_length,
    max_prediction_length=max_prediction_length,
    static_categoricals=["Symbol"],
    static_reals=[],
    time_varying_known_categoricals=[
        "hour_of_day",
        "day_of_week",
    ],
    time_varying_known_reals=[
        "time_idx",
    ],
    time_varying_unknown_categoricals=[],
    time_varying_unknown_reals=["V1", "V2","V3", "T", "V4"],
    constant_fill_strategy={"T": 0},
    dropout_categoricals=[],
)
print('Max Prediction Index : ',training.index.time.max())
validation = TimeSeriesDataSet.from_dataset(training, data, min_prediction_idx=training.index.time.max()+1)
batch_size = 128
train_dataloader = training.to_dataloader(train=True, batch_size=batch_size, num_workers=1)
val_dataloader = validation.to_dataloader(train=False, batch_size=batch_size, num_workers=1)

print(len(training), len(validation))
print(len(train_dataloader), len(val_dataloader))`

This is what gets printed by the code :

Len of training data is : 25920
Len of val data is : 2880
Min Prediction Index Value is 25919

25920 1
202 1

You can see that the training dataset is good and the batches are also okay but validation batch is also 1 and dataset length is also 1.

One more thing. if i use predict = False it generates validation data correctly but another bug arises due to that. if i use predict = True, only 1 batch and 1 sequence is given if i use predict_mode = true on the training dataset it also generates only 1 batch.

Here is a sample of my CSV sample_data.csv.zip

Please Help

• PyTorch-Forecasting version: 0.7
• PyTorch version: 1.7.0
• Python version:3.8.3
• Operating System:windows 10

Hi, I'm trying to follow the tutorial with my own data. When I run the learning rate finder, i got this error:

here is the full traceback:

RuntimeError                              Traceback (most recent call last)
<ipython-input-26-a92b5627800b> in <module>
      1 # find optimal learning rate
----> 2 res = trainer.tuner.lr_find(
      3     tft,
      4     train_dataloader=train_dataloader,
      5     val_dataloaders=val_dataloader,

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\tuner\tuning.py in lr_find(self, model, train_dataloader, val_dataloaders, min_lr, max_lr, num_training, mode, early_stop_threshold, datamodule)
    118             datamodule: Optional[LightningDataModule] = None
    119     ):
--> 120         return lr_find(
    121             self.trainer,
    122             model,

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\tuner\lr_finder.py in lr_find(trainer, model, train_dataloader, val_dataloaders, min_lr, max_lr, num_training, mode, early_stop_threshold, datamodule)
    167 
    168     # Fit, lr & loss logged in callback
--> 169     trainer.fit(model,
    170                 train_dataloader=train_dataloader,
    171                 val_dataloaders=val_dataloaders,

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\trainer\trainer.py in fit(self, model, train_dataloader, val_dataloaders, datamodule)
    444         self.call_hook('on_fit_start')
    445 
--> 446         results = self.accelerator_backend.train()
    447         self.accelerator_backend.teardown()
    448 

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\accelerators\cpu_accelerator.py in train(self)
     57 
     58         # train or test
---> 59         results = self.train_or_test()
     60         return results
     61 

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\accelerators\accelerator.py in train_or_test(self)
     64             results = self.trainer.run_test()
     65         else:
---> 66             results = self.trainer.train()
     67         return results
     68 

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\trainer\trainer.py in train(self)
    493 
    494                 # run train epoch
--> 495                 self.train_loop.run_training_epoch()
    496 
    497                 if self.max_steps and self.max_steps <= self.global_step:

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\trainer\training_loop.py in run_training_epoch(self)
    559             # TRAINING_STEP + TRAINING_STEP_END
    560             # ------------------------------------
--> 561             batch_output = self.run_training_batch(batch, batch_idx, dataloader_idx)
    562 
    563             # when returning -1 from train_step, we end epoch early

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\trainer\training_loop.py in run_training_batch(self, batch, batch_idx, dataloader_idx)
    726 
    727                         # optimizer step
--> 728                         self.optimizer_step(optimizer, opt_idx, batch_idx, train_step_and_backward_closure)
    729 
    730                     else:

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\trainer\training_loop.py in optimizer_step(self, optimizer, opt_idx, batch_idx, train_step_and_backward_closure, *args, **kwargs)
    467         with self.trainer.profiler.profile("optimizer_step"):
    468             # optimizer step lightningModule hook
--> 469             self.trainer.accelerator_backend.optimizer_step(
    470                 optimizer, batch_idx, opt_idx, train_step_and_backward_closure, *args, **kwargs
    471             )

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\accelerators\accelerator.py in optimizer_step(self, optimizer, batch_idx, opt_idx, lambda_closure, *args, **kwargs)
    112 
    113         # model hook
--> 114         model_ref.optimizer_step(
    115             epoch=self.trainer.current_epoch,
    116             batch_idx=batch_idx,

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\core\lightning.py in optimizer_step(self, epoch, batch_idx, optimizer, optimizer_idx, optimizer_closure, on_tpu, using_native_amp, using_lbfgs, *args, **kwargs)
   1378             optimizer.step(*args, **kwargs)
   1379         else:
-> 1380             optimizer.step(closure=optimizer_closure, *args, **kwargs)
   1381 
   1382     def optimizer_zero_grad(

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\torch\optim\lr_scheduler.py in wrapper(*args, **kwargs)
     65                 instance._step_count += 1
     66                 wrapped = func.__get__(instance, cls)
---> 67                 return wrapped(*args, **kwargs)
     68 
     69             # Note that the returned function here is no longer a bound method,

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_forecasting\optim.py in step(self, closure)
    129             closure: A closure that reevaluates the model and returns the loss.
    130         """
--> 131         _ = closure()
    132         loss = None
    133         # note - below is commented out b/c I have other work that passes back

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\trainer\training_loop.py in train_step_and_backward_closure()
    716 
    717                         def train_step_and_backward_closure():
--> 718                             result = self.training_step_and_backward(
    719                                 split_batch,
    720                                 batch_idx,

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\trainer\training_loop.py in training_step_and_backward(self, split_batch, batch_idx, opt_idx, optimizer, hiddens)
    821             # backward pass
    822             with self.trainer.profiler.profile("model_backward"):
--> 823                 self.backward(result, optimizer, opt_idx)
    824 
    825             # hook - call this hook only

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\trainer\training_loop.py in backward(self, result, optimizer, opt_idx, *args, **kwargs)
    841             self.trainer.accelerator_backend.backward(result, optimizer, opt_idx, *args, **kwargs)
    842         else:
--> 843             result.closure_loss = self.trainer.accelerator_backend.backward(
    844                 result.closure_loss, optimizer, opt_idx, *args, **kwargs
    845             )

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\accelerators\accelerator.py in backward(self, closure_loss, optimizer, opt_idx, *args, **kwargs)
     93             # do backward pass
     94             model = self.trainer.get_model()
---> 95             model.backward(closure_loss, optimizer, opt_idx, *args, **kwargs)
     96 
     97             # once backward has been applied, release graph

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\pytorch_lightning\core\lightning.py in backward(self, loss, optimizer, optimizer_idx, *args, **kwargs)
   1256         """
   1257         if self.trainer.train_loop.automatic_optimization or self._running_manual_backward:
-> 1258             loss.backward(*args, **kwargs)
   1259 
   1260     def toggle_optimizer(self, optimizer: Optimizer, optimizer_idx: int):

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\torch\tensor.py in backward(self, gradient, retain_graph, create_graph)
    219                 retain_graph=retain_graph,
    220                 create_graph=create_graph)
--> 221         torch.autograd.backward(self, gradient, retain_graph, create_graph)
    222 
    223     def register_hook(self, hook):

c:\users\u2\appdata\local\programs\python\python38\lib\site-packages\torch\autograd\__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
    128         retain_graph = create_graph
    129 
--> 130     Variable._execution_engine.run_backward(
    131         tensors, grad_tensors_, retain_graph, create_graph,
    132         allow_unreachable=True)  # allow_unreachable flag

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

Presently, the hyperparameter optimization returns the best parameters for

{'gradient_clip_val': 1.028043566346387,
 'hidden_size': 176,
 'dropout': 0.22095396475678628,
 'hidden_continuous_size': 36,
 'attention_head_size': 1}

even when use_learning_rate_finder=True. I expect this could be fixed easily just by adding it to the optuna parameters to track.

Bumps sphinx from 5.3.0 to 6.1.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps ipython from 8.6.0 to 8.8.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps nbsphinx from 0.8.10 to 0.8.11.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

All of the versions of pytorch-forecasting on conda-forge >= 0.6.0 have a requirement of prettytable <0.8. However, looking at pyproject.toml, it does not appear that prettytable of any version is a dependency of this package. Is the conda-forge requirement valid, and if so, would it be possible to support a newer version of prettytable?

Bumps setuptools from 65.3.0 to 65.5.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

• PyTorch-Forecasting version: 0.10.3
• PyTorch version:
• Python version: 3.10
• Operating System: Microsoft Windows 10

hey :) How can I create train, validation, and test sets for a time series dataset with multiple series, where each series has a different length (maximum time index)? In particular, how can I specify the starting point for the prediction horizon for each series, since the min_prediction_length parameter of the DeepAR model expects an integer value that is applicable to all series?

Actual behavior

However, result was .... I think it has to do with ... because of ...

Code to reproduce the problem

Paste the command(s) you ran and the output. Including a link to a colab notebook will speed up issue resolution. If there was a crash, please include the traceback here. The code used to initialize the TimeSeriesDataSet and model should be also included.