Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration of Tensorboard #635

Open
oguiza opened this issue Dec 5, 2022 Discussed in #633 · 0 comments
Open

Integration of Tensorboard #635

oguiza opened this issue Dec 5, 2022 Discussed in #633 · 0 comments
Labels
bug Something isn't working

Comments

@oguiza
Copy link
Contributor

oguiza commented Dec 5, 2022

Discussed in #633

Originally posted by Samcro5C November 30, 2022
When I tried to use the fastai callback TensorBoardCallback for a TSForecaster (minimal example from Readme), I ran into a RuntimeError (RuntimeError: Exception occured in TensorBoardCallback when calling event after_epoch: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.), so I was wondering if tsai supports tensorboard at all or if this is a bug. Thank you in advance.

Code to reproduce the issue:

X, y, splits = get_UCR_data('LSST', split_data=False)
tfms = [None, TSClassification()]
batch_tfms = TSStandardize(by_sample=True)
dls = get_ts_dls(X, y, splits=splits, tfms=tfms, batch_tfms=batch_tfms)
cbs = [TensorBoardCallback()]
learn = ts_learner(dls, InceptionTimePlus, metrics=accuracy, cbs=cbs)
learn.fit_one_cycle(10, 1e-2)

Here's the full traceback:

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

[<ipython-input-36-af203e69f537>](https://localhost:8080/#) in <module>
      5 cbs = [TensorBoardCallback()]
      6 learn = ts_learner(dls, InceptionTimePlus, metrics=accuracy, cbs=cbs)
----> 7 learn.fit_one_cycle(10, 1e-2)

18 frames

[/usr/local/lib/python3.8/dist-packages/fastai/callback/schedule.py](https://localhost:8080/#) in fit_one_cycle(self, n_epoch, lr_max, div, div_final, pct_start, wd, moms, cbs, reset_opt, start_epoch)
    117     scheds = {'lr': combined_cos(pct_start, lr_max/div, lr_max, lr_max/div_final),
    118               'mom': combined_cos(pct_start, *(self.moms if moms is None else moms))}
--> 119     self.fit(n_epoch, cbs=ParamScheduler(scheds)+L(cbs), reset_opt=reset_opt, wd=wd, start_epoch=start_epoch)
    120 
    121 # %% ../../nbs/14_callback.schedule.ipynb 50

[/usr/local/lib/python3.8/dist-packages/fastai/learner.py](https://localhost:8080/#) in fit(self, n_epoch, lr, wd, cbs, reset_opt, start_epoch)
    254             self.opt.set_hypers(lr=self.lr if lr is None else lr)
    255             self.n_epoch = n_epoch
--> 256             self._with_events(self._do_fit, 'fit', CancelFitException, self._end_cleanup)
    257 
    258     def _end_cleanup(self): self.dl,self.xb,self.yb,self.pred,self.loss = None,(None,),(None,),None,None

[/usr/local/lib/python3.8/dist-packages/fastai/learner.py](https://localhost:8080/#) in _with_events(self, f, event_type, ex, final)
    191 
    192     def _with_events(self, f, event_type, ex, final=noop):
--> 193         try: self(f'before_{event_type}');  f()
    194         except ex: self(f'after_cancel_{event_type}')
    195         self(f'after_{event_type}');  final()

[/usr/local/lib/python3.8/dist-packages/fastai/learner.py](https://localhost:8080/#) in _do_fit(self)
    243         for epoch in range(self.n_epoch):
    244             self.epoch=epoch
--> 245             self._with_events(self._do_epoch, 'epoch', CancelEpochException)
    246 
    247     def fit(self, n_epoch, lr=None, wd=None, cbs=None, reset_opt=False, start_epoch=0):

[/usr/local/lib/python3.8/dist-packages/fastai/learner.py](https://localhost:8080/#) in _with_events(self, f, event_type, ex, final)
    193         try: self(f'before_{event_type}');  f()
    194         except ex: self(f'after_cancel_{event_type}')
--> 195         self(f'after_{event_type}');  final()
    196 
    197     def all_batches(self):

[/usr/local/lib/python3.8/dist-packages/fastai/learner.py](https://localhost:8080/#) in __call__(self, event_name)
    169 
    170     def ordered_cbs(self, event): return [cb for cb in self.cbs.sorted('order') if hasattr(cb, event)]
--> 171     def __call__(self, event_name): L(event_name).map(self._call_one)
    172 
    173     def _call_one(self, event_name):

[/usr/local/lib/python3.8/dist-packages/fastcore/foundation.py](https://localhost:8080/#) in map(self, f, gen, *args, **kwargs)
    154     def range(cls, a, b=None, step=None): return cls(range_of(a, b=b, step=step))
    155 
--> 156     def map(self, f, *args, gen=False, **kwargs): return self._new(map_ex(self, f, *args, gen=gen, **kwargs))
    157     def argwhere(self, f, negate=False, **kwargs): return self._new(argwhere(self, f, negate, **kwargs))
    158     def argfirst(self, f, negate=False): return first(i for i,o in self.enumerate() if f(o))

[/usr/local/lib/python3.8/dist-packages/fastcore/basics.py](https://localhost:8080/#) in map_ex(iterable, f, gen, *args, **kwargs)
    838     res = map(g, iterable)
    839     if gen: return res
--> 840     return list(res)
    841 
    842 # %% ../nbs/01_basics.ipynb 336

[/usr/local/lib/python3.8/dist-packages/fastcore/basics.py](https://localhost:8080/#) in __call__(self, *args, **kwargs)
    823             if isinstance(v,_Arg): kwargs[k] = args.pop(v.i)
    824         fargs = [args[x.i] if isinstance(x, _Arg) else x for x in self.pargs] + args[self.maxi+1:]
--> 825         return self.func(*fargs, **kwargs)
    826 
    827 # %% ../nbs/01_basics.ipynb 326

[/usr/local/lib/python3.8/dist-packages/fastai/learner.py](https://localhost:8080/#) in _call_one(self, event_name)
    173     def _call_one(self, event_name):
    174         if not hasattr(event, event_name): raise Exception(f'missing {event_name}')
--> 175         for cb in self.cbs.sorted('order'): cb(event_name)
    176 
    177     def _bn_bias_state(self, with_bias): return norm_bias_params(self.model, with_bias).map(self.opt.state)

[/usr/local/lib/python3.8/dist-packages/fastai/callback/core.py](https://localhost:8080/#) in __call__(self, event_name)
     60             try: res = getcallable(self, event_name)()
     61             except (CancelBatchException, CancelBackwardException, CancelEpochException, CancelFitException, CancelStepException, CancelTrainException, CancelValidException): raise
---> 62             except Exception as e: raise modify_exception(e, f'Exception occured in `{self.__class__.__name__}` when calling event `{event_name}`:\n\t{e.args[0]}', replace=True)
     63         if event_name=='after_fit': self.run=True #Reset self.run to True at each end of fit
     64         return res

[/usr/local/lib/python3.8/dist-packages/fastai/callback/core.py](https://localhost:8080/#) in __call__(self, event_name)
     58         res = None
     59         if self.run and _run:
---> 60             try: res = getcallable(self, event_name)()
     61             except (CancelBatchException, CancelBackwardException, CancelEpochException, CancelFitException, CancelStepException, CancelTrainException, CancelValidException): raise
     62             except Exception as e: raise modify_exception(e, f'Exception occured in `{self.__class__.__name__}` when calling event `{event_name}`:\n\t{e.args[0]}', replace=True)

[/usr/local/lib/python3.8/dist-packages/fastai/callback/tensorboard.py](https://localhost:8080/#) in after_epoch(self)
     74             preds = getcallable(self.loss_func, 'activation')(self.pred)
     75             out = getcallable(self.loss_func, 'decodes')(preds)
---> 76             x,y,its,outs = self.dls.valid.show_results(b, out, show=False, max_n=self.n_preds)
     77             tensorboard_log(x, y, its, outs, self.writer, self.train_iter)
     78 

[/usr/local/lib/python3.8/dist-packages/tsai/data/core.py](https://localhost:8080/#) in show_results(self, b, preds, ctxs, max_n, nrows, ncols, figsize, **kwargs)
    643         p = self.decode_batch((b[0],preds), max_n=max_n)
    644         if figsize is None: figsize = (ncols*6, max_n//ncols*4)
--> 645         if ctxs is None: ctxs = get_grid(min(len(t), nrows*ncols), nrows=None, ncols=ncols, figsize=figsize, **kwargs)
    646         for i,ctx in enumerate(ctxs):
    647             title = f'True: {t[i][1]}\nPred: {p[i][1]}'

[/usr/local/lib/python3.8/dist-packages/fastai/vision/data.py](https://localhost:8080/#) in get_grid(n, nrows, ncols, figsize, double, title, return_fig, flatten, **kwargs)
     34         ncols = int(np.ceil(n/nrows))
     35     if double: ncols*=2 ; n*=2
---> 36     fig,axs = subplots(nrows, ncols, figsize=figsize, **kwargs)
     37     if flatten: axs = [ax if i<n else ax.set_axis_off() for i, ax in enumerate(axs.flatten())][:n]
     38     if title is not None: fig.suptitle(title, weight='bold', size=14)

[/usr/local/lib/python3.8/dist-packages/fastai/torch_core.py](https://localhost:8080/#) in subplots(nrows, ncols, figsize, imsize, suptitle, **kwargs)
     51         h=nrows*imsize if suptitle is None or imsize>2 else nrows*imsize+0.6 #https://github.com/matplotlib/matplotlib/issues/5355
     52         figsize=(ncols*imsize, h)
---> 53     fig,ax = plt.subplots(nrows, ncols, figsize=figsize, **kwargs)
     54     if suptitle is not None: fig.suptitle(suptitle)
     55     if nrows*ncols==1: ax = array([ax])

[/usr/local/lib/python3.8/dist-packages/matplotlib/pyplot.py](https://localhost:8080/#) in subplots(nrows, ncols, sharex, sharey, squeeze, subplot_kw, gridspec_kw, **fig_kw)
   1175 
   1176     """
-> 1177     fig = figure(**fig_kw)
   1178     axs = fig.subplots(nrows=nrows, ncols=ncols, sharex=sharex, sharey=sharey,
   1179                        squeeze=squeeze, subplot_kw=subplot_kw,

[/usr/local/lib/python3.8/dist-packages/matplotlib/pyplot.py](https://localhost:8080/#) in figure(num, figsize, dpi, facecolor, edgecolor, frameon, FigureClass, clear, **kwargs)
    538             dpi = 72
    539 
--> 540         figManager = new_figure_manager(num, figsize=figsize,
    541                                         dpi=dpi,
    542                                         facecolor=facecolor,

[/usr/local/lib/python3.8/dist-packages/matplotlib/backend_bases.py](https://localhost:8080/#) in new_figure_manager(cls, num, *args, **kwargs)
   3355         from matplotlib.figure import Figure
   3356         fig_cls = kwargs.pop('FigureClass', Figure)
-> 3357         fig = fig_cls(*args, **kwargs)
   3358         return cls.new_figure_manager_given_figure(num, fig)
   3359 

TypeError: Exception occured in `TensorBoardCallback` when calling event `after_epoch`:
	__init__() got an unexpected keyword argument 'show'
@oguiza oguiza added the bug Something isn't working label Dec 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant