精华内容
下载资源
问答
  • pytorch 预测_介绍pytorch预测

    千次阅读 2020-10-11 00:15:11
    pytorch 预测重点(Top highlight)I am pleased to announce the open-source Python package PyTorch Forecasting. It makes time series forecasting with neural networks simple both for data science ...

    pytorch 预测

    重点(Top highlight)

    I am pleased to announce the open-source Python package PyTorch Forecasting. It makes time series forecasting with neural networks simple both for data science practitioners and researchers.

    我很高兴地宣布开源Python软件包PyTorch Forecasting 。 对于数据科学从业者和研究人员而言,它使使用神经网络进行时间序列预测变得简单。

    为什么准确的预测如此重要? (Why is accurate forecasting so important?)

    Forecasting time series is important in many contexts and highly relevant to machine learning practitioners. Take, for example, demand forecasting from which many use cases derive. Almost every manufacturer would benefit from better understanding demand for their products in order to optimise produced quantities. Underproduce and you will lose revenues, overproduce and you will be forced to sell excess produce at a discount. Very related is pricing, which is essentially a demand forecast with a specific focus on price elasticity. Pricing is relevant to virtually all companies.

    预测时间序列在许多情况下都很重要,并且与机器学习从业者高度相关。 以需求预测为例,从中可以得出许多用例。 几乎每个制造商都将从对产品需求的了解中受益,从而优化产量。 生产不足,您将损失收入,生产过剩,您将被迫以折扣价出售多余的产品。 与定价息息相关的是定价,本质上是需求预测,特别关注价格弹性。 定价实际上与所有公司有关。

    For a large number of additional machine learning applications time is of the essence: predictive maintenance, risk scoring, fraud detection, etc. — you name it. The order of events and time between them is crucial to create a reliable forecast.

    对于大量其他机器学习应用程序来说,时间至关重要:预测性维护,风险评分,欺诈检测等-随便你说吧。 事件之间的顺序和时间对创建可靠的预测至关重要。

    In fact, while time series forecasting might not be as shiny as image recognition or language processing, it is more common in industry. This is because image recognition and language processing are relatively new to the field and are often used to power new products, while forecasting has been around for decades and sits at the heart of many decision (support) systems. The employment of high-accuracy machine learning models such as the ones in PyTorch Forecasting can better support decision making or even automate it, often directly resulting in multi-million dollars of additional profits.

    实际上,尽管时间序列预测可能不如图像识别或语言处理那么闪亮,但在行业中更为普遍。 这是因为图像识别和语言处理在该领域相对较新,并且经常用于驱动新产品,而预测已经存在了数十年,并且是许多决策(支持)系统的核心。 PyTorch预测中那样使用高精度的机器学习模型可以更好地支持决策甚至自动化,通常直接带来数百万美元的额外利润。

    深度学习成为强大的预测工具 (Deep learning emerges as a powerful forecasting tool)

    Deep learning (neural networks) has only recently outperformed traditional methods in time series forecasting, and has done so by a smaller margin than in image and language processing. In fact, in forecasting pure time series (which means without covariates, for example, price is to demand), deep learning has surpassed traditional statistical methods only two years ago [1]. However, as the field is quickly advancing, accuracy advantages associated with neural networks have become significant, which merits their increased use in time series forecasting. For example, the latest architecture N-BEATS demonstrates an 11% decrease in sMAPE on the M4 competition dataset compared to the next best non-neural-network-based method which is an ensemble of statistical methods [2]. This network is also implemented in PyTorch Forecasting.

    深度学习(神经网络)直到最近才在时间序列预测方面胜过传统方法,并且与图像和语言处理相比,其实现幅度较小。 实际上,在预测纯时间序列时(这意味着没有协变量,例如,价格是必需的),深度学习仅在两年前就超过了传统的统计方法[1]。 但是,随着该领域的快速发展,与神经网络相关的准确性优势变得十分重要,这应将其在时间序列预测中的应用日益广泛。 例如,最新的体系结构N-BEATS与第二种基于非神经网络的最佳统计方法相比,在M4竞争数据集上的sMAPE降低了11%[2]。 PyTorch预测中也实现了该网络。

    Furthermore, even compared to other popular machine learning algorithms, such as gradient boosted trees, deep learning has two advantages. First, neural network architectures can be designed with an inherent understanding of time, i.e. they automatically make a connection between temporally close data points. As a result, they can capture complex time dependencies. On the contrary, traditional machine learning models require manual creation of time series features, such as the average over the last x days. This diminishes the capabilities of these traditional machine learning algorithms to model time dependencies. Second, most tree-based models output a step function by design. Therefore, they cannot predict the marginal impact of change in inputs and, further, are notoriously unreliable in out-of-domain forecasts. For example, if we have observed only prices at 30 EUR and 50 EUR, tree-based models cannot assess the impact on demand of changing the price from 30 EUR to 35 EUR. In consequence, they often cannot directly be used to optimise inputs. However, this is often the whole point of creating a machine learning model — the value is in the optimisation of covariates. At the same time, neural networks employ continuous activation functions and are particularly good at interpolation in high-dimensional spaces, i.e. they can be used to optimise inputs, such as price.

    此外,即使与其他流行的机器学习算法(例如,梯度增强树)相比,深度学习也有两个优点。 首先,可以在对时间有内在理解的情况下设计神经网络体系结构,即,它们可以自动在时间上接近的数据点之间建立连接。 结果,它们可以捕获复杂的时间依赖性。 相反,传统的机器学习模型需要手动创建时间序列特征,例如最近x天的平均值。 这削弱了这些传统机器学习算法建模时间依赖性的能力。 其次,大多数基于树的模型都是根据设计输出阶跃函数的。 因此,他们无法预测输入变化的边际影响,而且,众所周知,在域外预测中也不可靠。 例如,如果我们仅观察到30欧元和50欧元的价格,则基于树的模型无法评估将价格从30欧元更改为35欧元对需求的影响。 因此,它们通常不能直接用于优化输入。 但是,这通常是创建机器学习模型的全部要点-价值在于优化协变量。 同时,神经网络采用连续激活函数,特别擅长在高维空间中进行插值,即它们可用于优化输入(例如价格)。

    什么是PyTorch预测? (What is PyTorch Forecasting?)

    Image for post

    PyTorch Forecasting aims to ease time series forecasting with neural networks for real-world cases and research alike. It does so by providing state-of-the-art time series forecasting architectures that can be easily trained with pandas dataframes.

    PyTorch Forecasting旨在通过神经网络简化针对实际案例和研究的时间序列预测。 通过提供最先进的时间序列预测体系结构,可以使用熊猫数据帧轻松对其进行培训

    • The high-level API significantly reduces workload for users because no specific knowledge is required on how to prepare a dataset for training with PyTorch. The TimeSeriesDataSet class takes care of variable transformations, missing values, randomised subsampling, multiple history lengths, etc. You only need to provide the pandas dataframe and specify from which variables a model should learn.

      高级API大大减少了用户的工作量,因为不需要任何有关如何准备数据集进行PyTorch培训的专门知识。 TimeSeriesDataSet类负责变量转换,缺失值,随机子采样,多个历史记录长度等。您只需要提供pandas数据框并指定模型应该从哪些变量中学习即可。

    • The BaseModel class provides generic visualisations such as showing predictions vs actuals and partial dependency plots. Training progress in the form of metrics and examples can be logged automatically in tensorboard.

      BaseModel类提供了通用的可视化效果,例如显示预测与实际情况以及部分依赖图。 可以将度量和示例形式的培训进度自动记录在tensorboard中

    • State-of-the-art networks are implemented for forecasting with and without covariates. They also come with dedicated in-built interpretation capabilities. For example, the Temporal Fusion Transformer [3], which has beaten Amazon’s DeepAR by 36–69% in benchmarks, comes with variable and time importance measures. See more on this in the example below.

      实现了具有和不具有协变量的最新网络以进行预测。 它们还具有专用的内置解释功能。 例如, Temporal Fusion Transformer [3]在基准测试上击败了亚马逊的DeepAR达36–69%,它具有可变性和时间重要性指标。 在下面的示例中查看更多信息。

    • A number of multi-horizon time series metrics exist to evaluate predictions over multiple prediction horizons.

      存在许多多水平时间序列度量以评估多个预测范围内的预测。
    • For scalability, the networks are designed to work with PyTorch Lightning which allows training on CPUs and single and multiple (distributed) GPUs out-of-the-box. The Ranger optimiser is implemented for faster model training.

      为了实现可扩展性,这些网络旨在与PyTorch Lightning配合使用,从而可以直接使用CPU以及单个和多个(分布式)GPU进行培训。 实施了Ranger优化器以加快模型训练的速度。

    • To facilitate experimentation and research, adding networks is straightforward. The code has been explicitly designed with PyTorch experts in mind. They will find it easy to implement even complex ideas. In fact, one has only to inherit from the BaseModel class and follow a convention for the forward’s method input and output, in order to immediately enable logging and interpretation capabilities.

      为了促进实验和研究,添加网络非常简单。 该代码已明确考虑了PyTorch专家的要求。 他们会发现,即使实施复杂的构想也很容易。 实际上,为了立即启用日志记录和解释功能,仅必须继承自BaseModel类并遵循前向方法输入和输出的约定。

    To get started, detailed tutorials in the documentation showcase end-to-end workflows. I will also discuss a concrete example later in this article.

    首先,文档中的详细教程展示了端到端的工作流程。 我还将在本文后面讨论一个具体示例。

    我们为什么需要这个包裹? (Why do we need this package?)

    PyTorch Forecasting helps overcome important barriers to the usage of deep learning. While deep learning has become dominant in image and language processing, this is less so in time series forecasting. The field remains dominated by traditional statistical methods such as ARIMA and machine learning algorithms such as gradient boosting, with the odd exemption of a Bayesian model. The reasons why deep learning has not yet become mainstream in time series forecasting are two-fold, all of which can already be overcome:

    PyTorch预测有助于克服使用深度学习的重要障碍。 尽管深度学习已在图像和语言处理中占主导地位,但在时间序列预测中却不那么重要。 该领域仍然由传统统计方法(如ARIMA)和机器学习算法(如梯度增强)以及贝叶斯模型的奇数豁免所主导。 深度学习尚未在时间序列预测中成为主流的原因有两个,所有这些都可以克服:

    1. Training neural networks almost always require GPUs which are not always readily available. Hardware requirements are often an important impediment. However, by moving computation into the cloud this hurdle can be overcome.

      训练神经网络几乎总是需要GPU,但这些GPU并非总是可以立即获得。 硬件要求通常是一个重要的障碍。 但是,通过将计算移动到云中,可以克服这一障碍。
    2. Neural networks are comparably harder to use than traditional methods. This is particularly the case for time series forecasting. There is a lack of a high-level API that works with the popular frameworks, such as PyTorch by Facebook or Tensorflow by Google. For traditional machine learning the sci-kit learn ecosystem exists which provides a standardised interface for practitioners.

      与传统方法相比,神经网络更难使用。 时间序列预测尤其如此。 缺少与流行框架兼容的高级API,例如Facebook的PyTorch或Google的Tensorflow。 对于传统的机器学习,存在sci-kit学习生态系统,该生态系统为从业人员提供了标准化的界面。

    This third hurdle is considered crucial in the deep learning community given its user-unfriendliness requires substantial software engineering. The following tweet summarises the sentiment of many:

    考虑到它的第三个障碍在深度学习社区中至关重要,因为它的用户不友好要求大量的软件工程。 以下推文总结了许多人的观点:

    Typical sentiment from a deep learning practitioner
    深度学习从业者的典型观点

    Some even thought the statement was trivial:

    有人甚至认为该声明是微不足道的:

    In a nutshell, PyTorch Forecasting aims to do what fast.ai has done for image recognition and natural language processing. That is significantly contributing to the proliferation of neural networks from academia into the real world. PyTorch Forecasting seeks to do the equivalent for time series forecasting by providing a high-level API for PyTorch that can directly make use of pandas dataframes. To facilitate learning it, unlike fast.ai, the package does not create a completely new API but rather builds on the well-established PyTorch and PyTorch Lightning APIs.

    简而言之, PyTorch Forecasting旨在做fast.ai在图像识别和自然语言处理方面所做的工作。 这极大地促进了神经网络从学术界向现实世界的扩散。 PyTorch预测通过为PyTorch提供可以直接利用熊猫数据帧的高级API来尝试进行时间序列预测。 为了方便学习,与fast.ai不同,该软件包不会创建全新的API,而是在完善的PyTorchPyTorch Lightning API的基础上构建。

    如何使用PyTorch预测? (How to use PyTorch Forecasting?)

    This small example showcases the power of the package and its most important abstractions. We will

    这个小例子展示了程序包的功能及其最重要的抽象。 我们会

    1. create a training and validation dataset,

      创建训练和验证数据集,
    2. train the Temporal Fusion Transformer [2]. This is an architecture developed by Oxford University and Google that has beaten Amazon’s DeepAR by 36–69% in benchmarks,

      训练时间融合变压器[2]。 这是由牛津大学和Google开发的架构,在基准测试中已经击败了亚马逊的DeepAR 36-69%,

    3. inspect results on the validation set and interpret the trained model.

      检查验证集上的结果并解释经过训练的模型。

    NOTE: The code below works only up to version 0.4.1 of PyTorch Forecasting and 0.9.0 of PyTorch Lightning. Minimal modifications are required to run with the latest version. A full tutorial with the latest code can be found here.

    注意:以下代码仅适用于PyTorch Forecasting的0.4.1版本和PyTorch Lightning的0.9.0版本。 要使用最新版本,需要进行最小的修改。 完整的教程以及最新的代码可以在这里找到。

    创建用于训练和验证的数据集 (Creating datasets for training and validation)

    First, we need to transform our time series into a pandas dataframe where each row can be identified with a time step and a time series. Fortunately, most datasets are already in this format. For this tutorial, we will use the Stallion dataset from Kaggle describing sales of various beverages. Our task is to make a six-month forecast of the sold volume by stock keeping units (SKU), that is products, sold by an agency, that is a store. There are about 21 000 monthly historic sales records. In addition to historic sales we have information about the sales price, the location of the agency, special days such as holidays, and volume sold in the entire industry.

    首先,我们需要将时间序列转换为pandas数据帧,在该数据帧中可以使用时间步长和时间序列来标识每一行。 幸运的是,大多数数据集已经采用这种格式。 在本教程中,我们将使用KaggleStallion数据集来描述各种饮料的销售。 我们的任务是通过库存单位(SKU)对销售量进行六个月的预测,SKU是由代理商(即商店)出售的产品。 每月大约有21000个历史销售记录。 除了历史销售外,我们还提供有关销售价格,代理商位置,特殊日子(如假期)和整个行业销售量的信息。

    from pytorch_forecasting.data.examples import get_stallion_datadata = get_stallion_data()  # load data as pandas dataframe

    The dataset is already in the correct format but misses some important features. Most importantly, we need to add a time index that is incremented by one for each time step. Further, it is beneficial to add date features, which in this case means extracting the month from the date record.

    数据集已经采用正确的格式,但是缺少一些重要功能。 最重要的是,我们需要添加一个时间索引,每个时间步长将其增加一个。 此外,添加日期功能是有益的,在这种情况下,这意味着从日期记录中提取月份。

    # add time index
    data["time_idx"] = data["date"].dt.year * 12 + data["date"].dt.monthdata["time_idx"] -= data["time_idx"].min()# add additional features
    # categories have to be strings
    data["month"] = data.date.dt.month.astype(str).astype("category")
    data["log_volume"] = np.log(data.volume + 1e-8)
    data["avg_volume_by_sku"] = (
    data
    .groupby(["time_idx", "sku"], observed=True)
    .
    volume.transform("mean"))
    data["avg_volume_by_agency"] = (
    data
    .groupby(["time_idx", "agency"], observed=True)
    .
    volume.transform("mean")
    )
    # we want to encode special days as one variable and
    # thus need to first reverse one-hot encoding
    special_days = [
    "easter_day", "good_friday", "new_year", "christmas",
    "labor_day", "independence_day", "revolution_day_memorial",
    "regional_games", "fifa_u_17_world_cup", "football_gold_cup",
    "beer_capital", "music_fest"
    ]data[special_days] = (
    data[special_days]
    .apply(lambda x: x.map({0: "-", 1: x.name}))
    .astype("category")
    )# show sample data
    data.sample(10, random_state=521)
    Image for post
    Random rows samples from dataframe
    来自数据框的随机行样本

    The next step is to convert the dataframe into a PyTorch Forecasting dataset. Apart from telling the dataset which features are categorical vs continuous and which are static vs varying in time, we also have to decide how we normalise the data. Here, we standard scale each time series separately and indicate that values are always positive.

    下一步是将数据转换为PyTorch预测数据集。 除了告诉数据集哪些特征是分类特征还是连续特征,哪些特征是静态特征还是随时间变化之外,我们还必须决定如何标准化数据。 在此,我们分别对每个时间序列进行标准缩放,并指示值始终为正。

    We also choose to use the last six months as a validation set.

    我们还选择将过去六个月用作验证集。

    from pytorch_forecasting.data import (
    TimeSeriesDataSet,
    GroupNormalizer
    )max_prediction_length = 6 # forecast 6 months
    max_encoder_length = 24 # use 24 months of history
    training_cutoff = data["time_idx"].max() - max_prediction_lengthtraining = TimeSeriesDataSet(
    data[lambda x: x.time_idx <= training_cutoff],
    time_idx="time_idx",
    target="volume",
    group_ids=["agency", "sku"],
    min_encoder_length=0, # allow predictions without history
    max_encoder_length=max_encoder_length,
    min_prediction_length=1,
    max_prediction_length=max_prediction_length,
    static_categoricals=["agency", "sku"],
    static_reals=[
    "avg_population_2017",
    "avg_yearly_household_income_2017"
    ],
    time_varying_known_categoricals=["special_days", "month"],
    # group of categorical variables can be treated as
    # one variable
    variable_groups={"special_days": special_days},
    time_varying_known_reals=[
    "time_idx",
    "price_regular",
    "discount_in_percent"
    ],
    time_varying_unknown_categoricals=[],
    time_varying_unknown_reals=[
    "volume",
    "log_volume",
    "industry_volume",
    "soda_volume",
    "avg_max_temp",
    "avg_volume_by_agency",
    "avg_volume_by_sku",
    ],
    target_normalizer=GroupNormalizer(
    groups=["agency", "sku"], coerce_positive=1.0
    ), # use softplus with beta=1.0 and normalize by group
    add_relative_time_idx=True, # add as feature
    add_target_scales=True, # add as feature
    add_encoder_length=True, # add as feature
    )# create validation set (predict=True) which means to predict the
    # last max_prediction_length points in time for each series
    validation = TimeSeriesDataSet.from_dataset(
    training, data, predict=True, stop_randomization=True
    )# create dataloaders for model
    batch_size = 128
    train_dataloader = training.to_dataloader(
    train=True, batch_size=batch_size, num_workers=0
    )
    val_dataloader = validation.to_dataloader(
    train=False, batch_size=batch_size * 10, num_workers=0
    )

    训练临时融合变压器 (Training the Temporal Fusion Transformer)

    It is now time to create our model. We train the model with PyTorch Lightning. Prior to training, you can identify the optimal learning rate with its learning rate finder (see the documentation for an example).

    现在是时候创建我们的模型了。 我们使用PyTorch Lightning训练模型。 在培训之前,您可以使用其学习率查找器确定最佳学习率(有关示例,请参阅文档)。

    import pytorch_lightning as pl
    from pytorch_lightning.callbacks import (
    EarlyStopping,
    LearningRateLogger
    )
    from pytorch_lightning.loggers import TensorBoardLogger
    from pytorch_forecasting.metrics import QuantileLoss
    from pytorch_forecasting.models import TemporalFusionTransformer# stop training, when loss metric does not improve on validation set
    early_stop_callback = EarlyStopping(
    monitor="val_loss",
    min_delta=1e-4,
    patience=10,
    verbose=False,
    mode="min"
    )
    lr_logger = LearningRateLogger() # log the learning rate
    logger = TensorBoardLogger("lightning_logs") # log to tensorboard# create trainer
    trainer = pl.Trainer(
    max_epochs=30,
    gpus=0, # train on CPU, use gpus = [0] to run on GPU
    gradient_clip_val=0.1,
    early_stop_callback=early_stop_callback,
    limit_train_batches=30, # running validation every 30 batches
    # fast_dev_run=True, # comment in to quickly check for bugs
    callbacks=[lr_logger],
    logger=logger,
    )# initialise model
    tft = TemporalFusionTransformer.from_dataset(
    training,
    learning_rate=0.03,
    hidden_size=16, # biggest influence network size
    attention_head_size=1,
    dropout=0.1,
    hidden_continuous_size=8,
    output_size=7, # QuantileLoss has 7 quantiles by default
    loss=QuantileLoss(),
    log_interval=10, # log example every 10 batches
    reduce_on_plateau_patience=4, # reduce learning automatically
    )
    tft.size() # 29.6k parameters in model# fit network
    trainer.fit(
    tft,
    train_dataloader=train_dataloader,
    val_dataloaders=val_dataloader
    )
    Image for post

    Training takes about three minutes on my Macbook but for larger networks and datasets, it can take hours. During training, we can monitor the tensorboard which can be spun up with tensorboard --logdir=lightning_logs. For example, we can monitor examples predictions on the training and validation set. As you can see from the figure below, forecasts look rather accurate. If you wonder, the grey lines denote the amount of attention the model pays to different points in time when making the prediction. This is a special feature of the Temporal Fusion Transformer.

    在Macbook上进行培训大约需要3分钟,但对于较大的网络和数据集,则可能需要数小时。 在培训过程中,我们可以监测tensorboard可与纺达tensorboard --logdir=lightning_logs 。 例如,我们可以监视训练和验证集上的示例预测。 从下图可以看到,预测看起来相当准确。 如果您想知道,灰线表示模型在进行预测时对不同时间点的关注程度。 这是时间融合变压器的一项特殊功能。

    Image for post
    Tensorboard panel showing training examples
    Tensorboard面板显示培训示例

    评估训练好的模型(Evaluating the trained model)

    After training, we can evaluate the metrics on the validation dataset and a couple of examples to see how well the model is doing. Given that we work with only 21 000 samples the results are very reassuring and can compete with results by a gradient booster.

    训练后,我们可以评估验证数据集上的指标和几个示例,以查看模型的运行情况。 鉴于我们仅处理21 000个样本,结果非常令人放心,并且可以通过梯度增强器与结果竞争。

    from pytorch_forecasting.metrics import MAE# load the best model according to the validation loss (given that
    # we use early stopping, this is not necessarily the last epoch)
    best_model_path = trainer.checkpoint_callback.best_model_path
    best_tft = TemporalFusionTransformer.load_from_checkpoint(best_model_path)# calculate mean absolute error on validation set
    actuals = torch.cat([y for x, y in iter(val_dataloader)])
    predictions = best_tft.predict(val_dataloader)MAE(predictions, actuals)
    Image for post

    Looking at the worst performers in terms of sMAPE gives us an idea where the model has issues with forecasting reliably. These examples can provide important pointers about how to improve the model. This kind of actuals vs predictions plots are available to all models.

    从sMAPE的角度来看效果最差的公司,可以使我们了解模型在可靠预测方面存在的问题。 这些示例可以提供有关如何改进模型的重要指示。 这种实际值与预测值图可用于所有模型。

    from pytorch_forecasting.metrics import SMAPE# calculate metric by which to display
    predictions, x = best_tft.predict(val_dataloader)
    mean_losses = SMAPE(reduction="none")(predictions, actuals).mean(1)
    indices = mean_losses.argsort(descending=True) # sort lossesraw_predictions, x = best_tft.predict(val_dataloader, mode="raw, return_x=True)# show only two examples for demonstration purposes
    for idx in range(2):
    best_tft.plot_prediction(
    x,
    raw_predictions,
    idx=indices[idx],
    add_loss_to_title=SMAPE()
    )
    Image for post
    The two worst predictions on the validation set. The white line is how much attention the transformer gives to a given point in time.
    验证集上的两个最差的预测。 白线是变压器对给定时间点的关注程度。

    Similarly, we could also visualise random examples from our model. Another feature of PyTorch Forecasting is interpretation of trained models. For example, all models allow us to readily calculate partial dependence plots. However, for brevity we will show here some inbuilt interpretation capabilities of the Temporal Fusion Transformer. It enables the variable importances by design of the neural network.

    同样,我们还可以可视化模型中的随机示例。 PyTorch预测的另一个功能是训练模型的解释。 例如,所有模型都使我们能够轻松计算偏倚图。 但是,为简便起见,我们将在此处展示Temporal Fusion Transformer的一些内置解释功能。 通过神经网络的设计,它可以实现可变的重要性。

    interpretation = best_tft.interpret_output(
    raw_predictions, reduction="sum"
    )best_tft.plot_interpretation(interpretation)
    Image for post

    Unsurprisingly, the past observed volume features as the top variable in the encoder and price related variables are among the top predictors in the decoder. Maybe more interesting is that the agency is ranked only fifth amongst the static variables. However, given that the second and third variable are related to location, we could expect agency to rank far higher if those two were not included in the model.

    毫不奇怪,过去观察到的交易量特征是编码器中的顶级变量和价格相关变量,是解码器中的顶级预测变量。 也许更有趣的是,该机构在静态变量中仅排名第五。 但是,鉴于第二个和第三个变量与位置有关,如果模型中未包含这两个变量,我们可以期望代理商的排名更高。

    概要 (Summary)

    It is very easy to train a model and get insights into its inner workings with PyTorch Forecasting. As a practitioner, you can employ the package to train and interpret state-of-the-art models out-of-the-box. With PyTorch Lightning integration training and prediction is scalable. As a researcher, you can leverage the package to get automatic tracking and introspection capabilities for your architecture and apply it seamlessly to multiple datasets.

    使用PyTorch预测非常容易训练模型并深入了解其内部工作原理。 作为一名从业者,您可以使用该软件包来立即训练和解释最新模型。 借助PyTorch Lightning,集成培训和预测是可扩展的。 作为研究人员,您可以利用该软件包为您的体系结构获得自动跟踪和自省功能,并将其无缝地应用于多个数据集。

    代码,文档以及贡献方式 (Code, documentation and how to contribute)

    The code for this tutorial can be found in this notebook: https://github.com/jdb78/pytorch-forecasting/blob/master/docs/source/tutorials/stallion.ipynb

    可以在以下笔记本中找到本教程的代码: https : //github.com/jdb78/pytorch-forecasting/blob/master/docs/source/tutorials/stallion.ipynb

    Install PyTorch Forecasting with

    使用安装PyTorch预测

    pip install pytorch-forecasting

    or

    要么

    conda install -c conda-forge pytorch-forecasting

    GitHub repository: https://github.com/jdb78/pytorch-forecasting

    GitHub存储库: https : //github.com/jdb78/pytorch-forecasting

    Documentation (including tutorials): https://pytorch-forecasting.readthedocs.io

    文档(包括教程): https : //pytorch-forecasting.readthedocs.io

    The package is open source under the MIT Licence which permits commercial use. Contributions are very welcome! Please read the contribution guidelines upfront to ensure your contribution is merged swiftly.

    该软件包是根据MIT许可开放的,允许商业使用。 欢迎捐款! 请先阅读贡献准则,以确保您的贡献被Swift合并。

    相关工作 (Related Work)

    Gluon-TS by Amazon aims to provide a similar interface but has two distinct disadvantages compared to PyTorch Forecasting. First, the package’s backend is MXNet, a deep learning framework trailing in popularity PyTorch and Tensorflow. Second, while it is a powerful framework, it can be difficult to master and modify given its complex object inheritance structure and tight coupling of components.

    亚马逊的Gluon -TS旨在提供类似的界面,但与PyTorch预测相比有两个明显的缺点。 首先,该软件包的后端是MXNet ,这是一种流行于PyTorchTensorflow的深度学习框架。 其次,尽管它是一个功能强大的框架,但鉴于其复杂的对象继承结构和组件的紧密耦合,可能难以掌握和修改。

    翻译自: https://towardsdatascience.com/introducing-pytorch-forecasting-64de99b9ef46

    pytorch 预测

    展开全文
  • Pytorch Forecasting旨在通过神经网络简化实际案例和研究中的时间序列预测。 具体来说,该软件包提供了有关“迈向数据科学”的文章,介绍了该软件包并提供了背景信息。 Pytorch Forecasting旨在通过神经网络简化实际...
  • pytorch预测时间序列Flow Forecast is a recently created open-source framework that aims to make it easy to use state of the art machine learning models to forecast and/or classify complex temporal data...

    pytorch预测时间序列

    Flow Forecast is a recently created open-source framework that aims to make it easy to use state of the art machine learning models to forecast and/or classify complex temporal data. Additionally, flow-forecast natively integrates with Google Cloud Platform, Weights and Biases, Colaboratory, and other tools commonly used in industry.

    Flow Forecast是最近创建的开源框架 ,旨在简化使用最先进的机器学习模型来预测和/或分类复杂的时间数据的过程。 此外,流程预测与Google Cloud Platform,权重和偏差,协作以及其他行业常用工具进行了本地集成。

    Background

    背景

    Image for post

    In some of my previous articles I talked about the need for accurate time series forecasts and the promise of using deep learning. Flow-Forecast was originally, created to forecast stream and river flows using variations of the transformer and baseline models. However, in the process of training the transformers I encountered several issues related to finding the right hyper-parameters and the right architecture. Therefore, it became necessary to develop a platform for trying out many configurations. Flow forecast is designed to allow you to very easily try out a number of different hyper-parameters and training options for your models. Changing a model is as simple as swapping out the model’s name in the configuration file.

    在之前的一些文章中,我谈到了对准确的时间序列预测的需求以及使用深度学习的希望 。 Flow-Forecast最初是使用变压器和基准线模型的变化来预测河流和河流流量的方法。 但是,在培训变压器的过程中,我遇到了一些与找到正确的超参数和正确的体系结构有关的问题。 因此,有必要开发一种平台来试用许多配置。 流量预测旨在让您非常轻松地为模型尝试许多不同的超参数和训练选项。 更改模型就像交换配置文件中的模型名称一样简单。

    Another problem I faced was how to integrate additional static datasets into the forecasts. For river flow forecasting, there was a lot of meta-data such as latitude, longitude, soil depth, elevation, slope, etc. For this, we decided to look into unsupervised methods like autoencoders for forming an embedding. This spurred the idea of creating a generic way to synthesize embedding with the temporal forecast.

    我面临的另一个问题是如何将其他静态数据集集成到预测中。 对于河流流量的预测,有很多元数据,例如纬度,经度,土壤深度,高程,坡度等。为此,我们决定研究诸如自动编码器之类的无监督方法来形成嵌入。 这激发了创建通用方法以将嵌入与时间预测进行综合的想法。

    Using flow forecast

    使用流量预测

    There are a couple easy resources to use to get started with flow-forecast. I recorded a brief introduction video back in May and there are also more detailed live-coding sessions you can follow. We also have a basic tutorial notebook that you can use to get a sense of how flow-forecast works on a basic problem. Additionally, there are also a lot more detailed notebooks that we use for our core COVID-19 predictions. Finally, we also have ReadTheDocs available for in depth documentation as well as our official wiki pages.

    有两个简单的资源可用于开始流程预测。 我在5月份录制了一个简短的介绍视频 ,还有更多详细的实时编码会议可供您关注。 我们还有一个基本的教程笔记本 ,您可以用来了解流量预测在基本问题上的工作方式。 此外,还有很多更详细的笔记本可用于我们的核心COVID-19预测。 最后,我们还有ReadTheDocs可用于深入文档以及我们的官方Wiki页面。

    Contributing

    贡献

    We always welcome new models and loss functions in our repository. We have created some simple documentation that describes the contribution process. There are also several live coding videos so you can see how to add new models to the repository yourself.

    我们始终欢迎存储库中提供新的模型和损失函数。 我们创建了一些简单的文档来描述贡献过程 。 还有一些实时编码视频,因此您可以了解如何自己向存储库添加新模型。

    Core goals of flow forecast and roadmap

    流量预测和路线图的核心目标

    • Provide a central repository of all the latest time series forecasting and classification models written in PyTorch and easy to extend.

      提供一个以PyTorch编写且易于扩展的所有最新时间序列预测和分类模型的中央存储库。
    • Make it easy to evaluate your model with a wide variety of loss functions, evaluation metrics as well as view the graphs of forecasted versus real values.

      使用各种损失函数,评估指标以及查看预测值与实际值的图形,可以轻松评估模型。
    • Test out a wide variety of hyper-parameters, loss functions, and slight architecture variations. Keep track of your experiments with Wandb reports and automatic weight saving to the cloud.

      测试各种超参数,损失函数和轻微的体系结构变化。 借助Wandb报告和自动将重量减轻到云端的方法来跟踪您的实验。
    • Synthesize static meta-data into your forecasts from many modalities such as text, image, numerical, geo-spatial, etc

      通过多种方式将静态元数据综合到您的预测中,例如文本,图像,数字,地理空间等
    • Allow you to utilize the latest transfer learning/pre-training techniques for time series to improve your forecast.

      允许您针对时间序列利用最新的迁移学习/预培训技术来改善预测。
    • Enable the easy deployment of time series models to production along with a simple to use continuous evaluation mechanism.

      使时间序列模型易于部署到生产,并具有易于使用的连续评估机制。
    • Easy integration with cloud providers such as GCP, AWS, and Azure and their related data storage solutions GCS, S3, Redshift, BigQuery, etc

      轻松与GCP,AWS和Azure等云提供商及其相关数据存储解决方案GCS,S3,Redshift,BigQuery等集成

    Questions/Feedback

    问题/反馈

    We appreciate questions and feedback on our repository. Feel free to open an issue if you encounter a bug or want an additional feature such as a model or a loss function.

    感谢您对我们存储库的问题和反馈。 如果遇到错误或想要其他功能(例如模型或损失函数),请随时提出问题。

    翻译自: https://towardsdatascience.com/flow-forecast-a-time-series-forecasting-library-built-in-pytorch-bbae7d8407b3

    pytorch预测时间序列

    展开全文
  • PyTorch项目代码与资源列表 本文收集了大量基于 PyTorch 实现的代码链接,其中有适用于深度学习新手的“入门指导系列”,也有适用于老司机的论文代码实现,包括 Attention Based CNN、A3C、WGAN等等。所有代码均...

    PyTorch项目代码与资源列表


    本文收集了大量基于 PyTorch 实现的代码链接,其中有适用于深度学习新手的“入门指导系列”,也有适用于老司机的论文代码实现,包括 Attention Based CNN、A3C、WGAN等等。所有代码均按照所属技术领域分类,包括机器视觉/图像相关、自然语言处理相关、强化学习相关等等。所以如果你打算入手这风行一世的 PyTorch 技术,那么就快快收藏本文吧!

    PyTorch 是什么?

    PyTorch即 Torch 的 Python 版本。Torch 是由 Facebook 发布的深度学习框架,因支持动态定义计算图,相比于 Tensorflow 使用起来更为灵活方便,特别适合中小型机器学习项目和深度学习初学者。但因为 Torch 的开发语言是Lua,导致它在国内一直很小众。所以,在千呼万唤下,PyTorch应运而生!PyTorch 继承了 Troch 的灵活特性,又使用广为流行的 Python 作为开发语言,所以一经推出就广受欢迎!


    目录:
    入门系列教程
    入门实例
    图像、视觉、CNN相关实现
    对抗生成网络、生成模型、GAN相关实现
    机器翻译、问答系统、NLP相关实现
    先进视觉推理系统
    深度强化学习相关实现
    通用神经网络高级应用


    1 入门系列教程

    1.PyTorch Tutorials
    https://github.com/MorvanZhou/PyTorch-Tutorial.git
    著名的“莫烦”PyTorch系列教程的源码。


    2.Deep Learning with PyTorch: a 60-minute blitz
    http://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html
    PyTorch官网推荐的由网友提供的60分钟教程,本系列教程的重点在于介绍PyTorch的基本原理,包括自动求导,神经网络,以及误差优化API。


    3.Simple examples to introduce PyTorch
    https://github.com/jcjohnson/pytorch-examples.git
    由网友提供的PyTorch教程,通过一些实例的方式,讲解PyTorch的基本原理。内容涉及Numpy、自动求导、参数优化、权重共享等。


    2 入门实例

    1.Ten minutes pyTorch Tutorial
    https://github.com/SherlockLiao/pytorch-beginner.git
    知乎上“十分钟学习PyTorch“系列教程的源码。

    2.Official PyTorch Examples
    https://github.com/pytorch/examples
    官方提供的实例源码,包括以下内容:
    MNIST Convnets
    Word level Language Modeling using LSTM RNNs
    Training Imagenet Classifiers with Residual Networks
    Generative Adversarial Networks (DCGAN)
    Variational Auto-Encoders
    Superresolution using an efficient sub-pixel convolutional neural network
    Hogwild training of shared ConvNets across multiple processes on MNIST
    Training a CartPole to balance in OpenAI Gym with actor-critic
    Natural Language Inference (SNLI) with GloVe vectors, LSTMs, and torchtext
    Time sequence prediction - create an LSTM to learn Sine waves

    3.PyTorch Tutorial for Deep Learning Researchers
    https://github.com/yunjey/pytorch-tutorial.git

    据说是提供给深度学习科研者们的PyTorch教程。教程中的每个实例的代码都控制在30行左右,简单易懂,内容如下:

    PyTorch Basics
    Linear Regression
    Logistic Regression
    Feedforward Neural Network
    Convolutional Neural Network
    Deep Residual Network
    Recurrent Neural Network
    Bidirectional Recurrent Neural Network
    Language Model (RNN-LM)
    Generative Adversarial Network
    Image Captioning (CNN-RNN)
    Deep Convolutional GAN (DCGAN)
    Variational Auto-Encoder
    Neural Style Transfer
    TensorBoard in PyTorch


    4.PyTorch-playground
    https://github.com/aaron-xichen/pytorch-playground.git
    PyTorch初学者的Playground,在这里针对一下常用的数据集,已经写好了一些模型,所以大家可以直接拿过来玩玩看,目前支持以下数据集的模型。
    mnist, svhn
    cifar10, cifar100
    stl10
    alexnet
    vgg16, vgg16_bn, vgg19, vgg19_bn
    resnet18, resnet34, resnet50, resnet101, resnet152
    squeezenet_v0, squeezenet_v1
    inception_v3


    3 图像、视觉、CNN相关实现

    1.PyTorch-FCN
    https://github.com/wkentaro/pytorch-fcn.git
    FCN(Fully Convolutional Networks implemented) 的PyTorch实现。

    2.Attention Transfer
    https://github.com/szagoruyko/attention-transfer.git
    论文 "Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer" 的PyTorch实现。

    3.Wide ResNet model in PyTorch
    https://github.com/szagoruyko/functional-zoo.git
    一个PyTorch实现的 ImageNet Classification 。

    4.CRNN for image-based sequence recognition
    https://github.com/bgshih/crnn.git
    这个是 Convolutional Recurrent Neural Network (CRNN) 的 PyTorch 实现。CRNN 由一些CNN,RNN和CTC组成,常用于基于图像的序列识别任务,例如场景文本识别和OCR。

    5.Scaling the Scattering Transform: Deep Hybrid Networks
    https://github.com/edouardoyallon/pyscatwave.git
    使用了“scattering network”的CNN实现,特别的构架提升了网络的效果。

    6.Conditional Similarity Networks (CSNs)
    https://github.com/andreasveit/conditional-similarity-networks.git
    《Conditional Similarity Networks》的PyTorch实现。

    7.Multi-style Generative Network for Real-time Transfer
    https://github.com/zhanghang1989/PyTorch-Style-Transfer.git
    MSG-Net 以及 Neural Style 的 PyTorch 实现。

    8.Big batch training
    https://github.com/eladhoffer/bigBatch.git
    《Train longer, generalize better: closing the generalization gap in large batch training of neural networks》的 PyTorch 实现。

    9.CortexNet
    https://github.com/e-lab/pytorch-CortexNet.git
    一个使用视频训练的鲁棒预测深度神经网络。

    10.Neural Message Passing for Quantum Chemistry
    https://github.com/priba/nmp_qc.git
    论文《Neural Message Passing for Quantum Chemistry》的PyTorch实现,好像是讲计算机视觉下的神经信息传递。


    4 对抗生成网络、生成模型、GAN相关实现

    1.Generative Adversarial Networks (GANs) in PyTorch
    https://github.com/devnag/pytorch-generative-adversarial-networks.git
    一个非常简单的由PyTorch实现的对抗生成网络

    2.DCGAN & WGAN with Pytorch
    https://github.com/chenyuntc/pytorch-GAN.git
    由中国网友实现的DCGAN和WGAN,代码很简洁。

    3.Official Code for WGAN
    https://github.com/martinarjovsky/WassersteinGAN.git
    WGAN的官方PyTorch实现。

    4.DiscoGAN in PyTorch
    https://github.com/carpedm20/DiscoGAN-pytorch.git
    《Learning to Discover Cross-Domain Relations with Generative Adversarial Networks》的 PyTorch 实现。

    5.Adversarial Generator-Encoder Network
    https://github.com/DmitryUlyanov/AGE.git
    《Adversarial Generator-Encoder Networks》的 PyTorch 实现。

    6.CycleGAN and pix2pix in PyTorch
    https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix.git
    图到图的翻译,著名的 CycleGAN 以及 pix2pix 的PyTorch 实现。

    7.Weight Normalized GAN
    https://github.com/stormraiser/GAN-weight-norm.git
    《On the Effects of Batch and Weight Normalization in Generative Adversarial Networks》的 PyTorch 实现。

    5 机器翻译、问答系统、NLP相关实现

    1.DeepLearningForNLPInPytorch
    https://github.com/rguthrie3/DeepLearningForNLPInPytorch.git

    一套以 NLP 为主题的 PyTorch 基础教程。本教程使用Ipython Notebook编写,看起来很直观,方便学习。

    2.Practial Pytorch with Topic RNN & NLP
    https://github.com/spro/practical-pytorch
    以 RNN for NLP 为出发点的 PyTorch 基础教程,分为“RNNs for NLP”和“RNNs for timeseries data”两个部分。
    3.PyOpenNMT: Open-Source Neural Machine Translation
    https://github.com/OpenNMT/OpenNMT-py.git
    一套由PyTorch实现的机器翻译系统。

    4.Deal or No Deal? End-to-End Learning for Negotiation Dialogues
    https://github.com/facebookresearch/end-to-end-negotiator.git
    Facebook AI Research 论文《Deal or No Deal? End-to-End Learning for Negotiation Dialogues》的 PyTorch 实现。

    5.Attention is all you need: A Pytorch Implementation
    https://github.com/jadore801120/attention-is-all-you-need-pytorch.git
    Google Research 著名论文《Attention is all you need》的PyTorch实现。

    6.Improved Visual Semantic Embeddings
    https://github.com/fartashf/vsepp.git
    一种从图像中检索文字的方法,来自论文:《VSE++: Improved Visual-Semantic Embeddings》。

    7.Reading Wikipedia to Answer Open-Domain Questions
    https://github.com/facebookresearch/DrQA.git
    一个开放领域问答系统DrQA的PyTorch实现。

    8.Structured-Self-Attentive-Sentence-Embedding
    https://github.com/ExplorerFreda/Structured-Self-Attentive-Sentence-Embedding.git
    IBM 与 MILA 发表的《A Structured Self-Attentive Sentence Embedding》的开源实现。


    6 先进视觉推理系统

    1.Visual Question Answering in Pytorch
    https://github.com/Cadene/vqa.pytorch.git
    一个PyTorch实现的优秀视觉推理问答系统,是基于论文《MUTAN: Multimodal Tucker Fusion for Visual Question Answering》实现的。项目中有详细的配置使用方法说明。

    2.Clevr-IEP
    https://github.com/facebookresearch/clevr-iep.git
    Facebook Research 论文《Inferring and Executing Programs for Visual Reasoning》的PyTorch实现,讲的是一个可以基于图片进行关系推理问答的网络。


    7 深度强化学习相关实现

    1.Deep Reinforcement Learning withpytorch & visdom

    https://github.com/onlytailei/pytorch-rl.git
    多种使用PyTorch实现强化学习的方法。

    2.Value Iteration Networks in PyTorch
    https://github.com/onlytailei/Value-Iteration-Networks-PyTorch.git
    Value Iteration Networks (VIN) 的PyTorch实现。

    3.A3C in PyTorch
    https://github.com/onlytailei/A3C-PyTorch.git
    Adavantage async Actor-Critic (A3C) 的PyTorch实现。


    8 通用神经网络高级应用

    1.PyTorch-meta-optimizer
    https://github.com/ikostrikov/pytorch-meta-optimizer.git
    论文《Learning to learn by gradient descent by gradient descent》的PyTorch实现。

    2.OptNet: Differentiable Optimization as a Layer in Neural Networks
    https://github.com/locuslab/optnet.git
    论文《Differentiable Optimization as a Layer in Neural Networks》的PyTorch实现。

    3.Task-based End-to-end Model Learning
    https://github.com/locuslab/e2e-model-learning.git
    论文《Task-based End-to-end Model Learning》的PyTorch实现。

    4.DiracNets
    https://github.com/szagoruyko/diracnets.git
    不使用“Skip-Connections”而搭建特别深的神经网络的方法。

    5.ODIN: Out-of-Distribution Detector for Neural Networks
    https://github.com/ShiyuLiang/odin-pytorch.git
    这是一个能够检测“分布不足”(Out-of-Distribution)样本的方法的PyTorch实现。当“true positive rate”为95%时,该方法将DenseNet(适用于CIFAR-10)的“false positive rate”从34.7%降至4.3%。

    6.Accelerate Neural Net Training by Progressively Freezing Layers
    https://github.com/ajbrock/FreezeOut.git
    一种使用“progressively freezing layers”来加速神经网络训练的方法。

    7.Efficient_densenet_pytorch
    https://github.com/gpleiss/efficient_densenet_pytorch.git
    DenseNets的PyTorch实现,优化以节省GPU内存。
    展开全文
  • 吴恩达卷积神经网络,第一周作业PyTorch版本代码(gpu-cpu通用) 1.PyCharm上运行的PyTorch项目 ...5.含训练、模型保存、模型加载、单个图片预测代码 6.里面保存了个已经在gpu上训练好的模型,下载后也可以自行训练
  • CSRNet-pytorch版本代码复现

    千次阅读 热门讨论 2020-09-04 17:22:37
    网络结构图如下所示: 关于VGG网络的详细介绍可以参考博客:使用pytorch搭建自己的网络之VGG class CSRNet(nn.Module): def __init__(self, load_weights=True): super(CSRNet, self).__init__() self.frontend_feat...

    一、概述

    在之前的博客中我有详细介绍过人群计数领域中密度图的生成方法,还有一篇CSRNet的论文学习笔记。
    人群计数之生成密度图
    论文学习笔记:CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes
    接下来我使用ShanghaiTech数据集对论文中提出的CSRNet模型进行了复现,其中关于数据增强等一些细节进行了省略,以方便初学者更快地了解人群计数相关的方法。

    二、数据集预处理

    本文使用ShanghaiTech part_B_final部分数据集对进行复现,由于没有对数据进行预处理和数据增强等操作,而part_A_final部分的图片尺寸大小不一样,所以在进行训练的时候只能设置批处理大小为1(batch_size=1)。
    关于数据预处理部分的代码请参考:人群计数之生成密度图

    在这里插入图片描述
    运行代码之后我们可以看到在原来存放标注文件的文件夹生成每张图片对应的ground_truth密度图(h5文件)。

    经过CSRNet网络模型之后输入图片的尺寸变为原来的1/8,所以生成的真实密度图大小为原图片尺寸的1/8,如果做了数据增强处理(例如裁剪、resize()操作等),需要重新生成一次真实密度图。

    三、构建CSRNet网络

    CSRnet网络模型主要分为前端和后端网络,前端网络采用的是剔除了全连接层的VGG-16,后端网络采用了空洞卷积的操作,论文中为了对比不同模型的效果,也是搭建了四种空洞率不同的后端网络。网络结构图如下所示:
    在这里插入图片描述
    关于VGG网络的详细介绍可以参考博客:使用pytorch搭建自己的网络之VGG

    class CSRNet(nn.Module):
        def __init__(self, load_weights=True):
            super(CSRNet, self).__init__()
            self.frontend_feat = [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 'M', 512, 512, 512]
            self.backend_feat = [512, 512, 512,256,128,64]
            self.frontend = make_layers(self.frontend_feat)
            self.backend = make_layers(self.backend_feat,in_channels = 512,dilation = True)
            self.output_layer = nn.Conv2d(64, 1, kernel_size=1)
    
            if load_weights:  #加载VGG16的预训练模型
                vgg16 = models.vgg16(pretrained = True)
                self._initialize_weights()  #初始化权重
                #加载torchvision中的预训练模型和参数后通过state_dict()方法提取参数
                for i in range(len(self.frontend.state_dict().items())):
                    list(self.frontend.state_dict().items())[i][1].data[:] = list(vgg16.state_dict().items())[i][1].data[:]
    
        def forward(self,x):
            x = self.frontend(x)
            x = self.backend(x)
            x = self.output_layer(x)
            return x
    
        def _initialize_weights(self):
            for m in self.modules():
                if isinstance(m, nn.Conv2d):
                    nn.init.normal_(m.weight, std=0.01)
                    if m.bias is not None:
                        nn.init.constant_(m.bias, 0)
                elif isinstance(m, nn.BatchNorm2d):
                    nn.init.constant_(m.weight, 1)
                    nn.init.constant_(m.bias, 0)
    

    这里为了方便起见,直接使用了预训练的VGG-16模型和参数。在初始化函数__init__中,首先调用_initialize_weights()函数对所有层的参数进行初始化,然后使用VGG16预训练好的参数对其中属于VGG的层初始化。

    最后补上make_layers()函数:

    def make_layers(cfg, in_channels = 3, batch_norm=False, dilation = False):
        if dilation:
            d_rate = 2
        else:
            d_rate = 1
        layers = []
        for v in cfg:
            if v == 'M':
                layers += [nn.MaxPool2d(kernel_size=2, stride=2)]
            else:
                conv2d = nn.Conv2d(in_channels, v, kernel_size=3, padding=d_rate,dilation = d_rate)
                if batch_norm:
                    layers += [conv2d, nn.BatchNorm2d(v), nn.ReLU(inplace=True)]
                else:
                    layers += [conv2d, nn.ReLU(inplace=True)]
                in_channels = v
        return nn.Sequential(*layers)
    

    论文中经过对比发现CSRNet-B的效果最好,所以后面的研究中均使用了结构B。这里要修改的话也很简单,注意修改d_rate 的值就好。

    四、训练和测试

    4.1 主函数

    def main():
        # defining the location of dataset
        root = '../dataset/ShanghaiTech_Crowd_Counting_Dataset/'
        part_A_train = os.path.join(root, 'part_A_final/train_data', 'images')
        part_A_test = os.path.join(root, 'part_A_final/test_data', 'images')
        part_B_train = os.path.join(root, 'part_B_final/train_data', 'images')
        part_B_test = os.path.join(root, 'part_B_final/test_data', 'images')
    
        # 参数设置
        batch_size = 8
        num_epochs = 100
        lr = 1e-7
        momentum = 0.95
        decay = 5 * 1e-4
    
        #加载数据集
        train_iter,train_num = load_local_dataset(part_B_train,batch_size)
        print('train_num=',train_num)
        test_iter,test_num = load_local_dataset(part_B_test, batch_size)
        print('test_num=',test_num)
    
        #使用所有GPU进行训练
        net = torch.nn.DataParallel(model.CSRNet()).cuda()
    
        #定义损失函数和优化器
        loss = torch.nn.MSELoss(reduction='sum').cuda()
        optimizer = torch.optim.SGD(net.parameters(), lr=lr, momentum=momentum, weight_decay=decay)
    
        for i in range(num_epochs):
            print('第%d次训练' % (i+1) )
            train(train_iter, net, loss, optimizer)
            torch.save(net.module.state_dict(), f'./checkpoint/CSRNet_{str(i+1).zfill(4)}.pt')#保存模型
            validate(test_iter, net,test_num)
    

    首先定义相关参数,然后调用load_local_dataset()函数加载数据集,最后进行训练和测试,并保存模型。由于博主有多块GPU,所以使用了多块GPU共同训练以增加训练速度。代码片段是torch.nn.DataParallel(model.CSRNet()),如果使用此行代码,在保存训练模型并调用模型的时候要注意,使用其他方法可能会报错。

    4.2 加载数据集

    # 首先继承上面的dataset类。然后在__init__()方法中得到图像的路径,将图像路径组成一个数组,这样在__getitim__()中就可以直接读取:
    class MyDataset(Dataset):  # 创建自己的类:MyDataset,这个类是继承的torch.utils.data.Dataset
        def __init__(self, img_paths, transform=None, target_transform=None):  # 初始化一些需要传入的参数
            super(MyDataset, self).__init__()  # 对继承自父类的属性进行初始化
            self.img_paths = img_paths
            self.transform = transform
            self.target_transform = target_transform
    
        def __getitem__(self, index):  # 这个方法是必须要有的,用于按照索引读取每个元素的具体内容
            img_path = self.img_paths[index] #根据索引获取图片路径
            img = Image.open(img_path).convert('RGB')
    
            gt_path = img_path.replace('.jpg', '.h5').replace('images', 'ground_truth')
            gt_file = h5py.File(gt_path,'r')
            target = np.asarray(gt_file['density'])
            target = cv2.resize(target, (target.shape[1] // 8, target.shape[0] // 8), interpolation=cv2.INTER_CUBIC) * 64
    
            if self.transform is not None:
                img = self.transform(img)  # 数据标签转换为Tensor
    
            return img, target  # return回哪些内容,那么我们在训练时循环读取每个batch时,就能获得哪些内容
    
        def __len__(self):  # 这个函数也必须要写,它返回的是数据集的长度,也就是多少张图片,要和loader的长度作区分
            return len(self.img_paths)
    

    继承dataset类并重写该函数,然后定义load_local_dataset()函数来加载数据集,返回的是数据集的迭代器和数据集大小。

    def load_local_dataset(path_sets, batch_size=8):
        img_paths = []
        path_sets=[path_sets]
        for path in path_sets:
            for img_path in glob.glob(os.path.join(path, '*.jpg')):
                img_paths.append(img_path)
        # 加载数据集
        datasets =MyDataset(img_paths, transform=transform)
        # 调用DataLoader来创建数据集的迭代器
        dataset_iter = DataLoader(dataset=datasets, batch_size=batch_size, shuffle=True)
        return dataset_iter,len(img_paths)
    

    附上图像的初始化操作代码:

    # 图像的初始化操作
    transform = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
    ])
    

    4.3 训练

    def train(train_iter, net, criterion, optimizer):
        #开始训练
        net.train()  # 启用 BatchNormalization 和 Dropout
        for (img, target) in train_iter:
            img = img.cuda()
            target = torch.unsqueeze(target, 1).cuda()
            output = net(img) #调用模型进行训练
    
            #损失函数
            loss = criterion(output, target)
            #反向传播
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
    

    这里没有什么复杂的操作,唯一要注意的是target通过torch.unsqueeze操作进行维度扩充,使得target和output的维度相同,能够计算损失函数。

    4.4 测试

    def validate(test_iter, net, dataset_num):
        net.eval() #开始测试
        mae = 0
        mse = 0
    
        for (img, target) in test_iter:
            img = img.cuda()
            target = torch.unsqueeze(target, 1).cuda()
            output = net(img)
    
            mae += abs(output.data.sum() - target.data.sum())
            mse += pow((output.data.sum() - target.data.sum()),2)
        mae = mae / dataset_num
        mse = pow((mse / dataset_num),0.5)
    
        print('MAE {mae:.3f} MSE {mse:.3f} '.format(mae = mae,mse = mse))
    

    为了追求代码的极致简单,这里也只计算了MAE和MSE。

    4.5 补充

    1、在训练代码中均使用了GPU进行训练,如果你想使用CPU训练的话,修改一下代码net = torch.nn.DataParallel(model.CSRNet()).cuda()并去掉所有.cuda()后缀即可。
    2、此blog仅提供部分代码和相关讲解,如果想要获取全部资源可以通过下面链接:
    https://download.csdn.net/download/qq_40356092/12811426
    博主也会提供复现代码的所有指导。

    五、测试单张图片

    以下内容的代码请下载完整资源包:CSRNet-pytorch

    原始图片:
    在这里插入图片描述
    真实密度图:
    在这里插入图片描述
    预测的密度图:
    在这里插入图片描述
    预测人数为: 19.395416
    真实人数为: 21.089602

    六、 Write in the end

    由于在复现的时候参考了大量的代码,所以和其他博主/教程中提供的代码可能有相似之处,但是因为时间久远,所以很难找到相应的参考链接,在这里就不提供了~

    如果你对以上内容有任何疑问,均可以在留言区进行留言评论,博主看到了一定会及时回复。

    展开全文
  • FasterRCNN-pytorch代码解析

    千次阅读 2018-12-13 13:52:31
    先介绍一下代码的结构分布吧 1、tain.py文件是训练的时候首先执行的文件,里面的函数有eval()评估函数,train()训练函数 2、trainer.py文件是网络的流图,关于如何forward,如何计算loss,如何反向计算,如何...
  • 翻译自外网,这是我觉得少数在做预测的实验 原文链接 时间序列数据,顾名思义,是一种随时间变化的数据类型。...在本文中,我们将使用PyTorch库,这是深度学习中最常用的Python库之一。在继续学习之
  • with torch.no_grad(): # 使用model进行预测代码 pass
  • 本文收集了大量基于 PyTorch 实现的代码链接,其中有适用于深度学习新手的“入门指导系列”,也有适用于老司机的论文代码实现,包括 Attention Based CNN、A3C、WGAN等等。所有代码均按照所属技术领域分类,包括机器...
  • 在使用pytorch框架进行开发时,有时需要根据预测结果在原图画点,下面是实现的方法: 具体代码的含义将通过注释的形式进行标注: img_r = cv2.imread("test21/room/1.jpg") # 读入图片 img1 = cv2....
  • 本文收录了大量的pytorch实现的源码。有入门级的例子说明,也有场景应用实例,更有论文源码的实现。总之,先给记下来。 本文涵盖以下部分: -入门系列教程 -入门实例 -图像,视觉,CNN相关实现 -GAN相关实现 -...
  • AI challenger 场景分类 PyTorch 测试代码

    千次阅读 2017-10-14 08:15:56
    - 另外需要注意places 365 cnn 是采取了保存和加载整个模型的办法,而我自己在用时只保存了权重,因此这里需要先导入pytorch官方的模型。 - 经在验证集上多重的比较,除了loss的问题其他都正常。 - 只是为了测试...
  • Pytorch代码Pytorch 迁移学习教程代码 逐句详解

    千次阅读 多人点赞 2019-11-13 16:33:47
    Pytorch 迁移学习教程代码 逐句详解 以下是Pytorch官网中提供的迁移学习图片分类任务的教程代码,本博客将会对其进行逐句对照解析,以便于读者能够理解。 教程介绍 在本教程中,主要介绍基于迁移学习的卷积神经...
  • convolutional LSTM(convLSTM)的pytorch版本代码实现

    万次阅读 多人点赞 2019-07-30 10:44:43
    convolutional LSTM(convLSTM)是《Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting》一文提出的,用于降水预测。这一网络结构,既考虑了输入之间的空间关联,也考虑了时序...
  • 先贴一下其它教程 ...2.pytorch版本yolov3训练自己的数据: https://www.cnblogs.com/pprp/p/10863496.html 这个链接最下方还有 详细cfg文件讲解:https://pprp.github.io/2018/09/20/YOLO%20cfg%E6%9...
  • 点击关注我哦一篇文章带你亲临Kaggle项目这篇文章将使用 Kaggle 的 Montréal 自行车道数据集(数据集下载地址:https://www.kaggle.com/pablomonleon/montreal-bike-lanes)来演示 PyTorch 线性回归模型,并用它来...
  • 作者:忆臻 (哈工大SCIR实验室在读博士生)魏福煊 哈工大英才实验班本科生谢天宝 哈工大英才实验班本科生一、前言在我们要用pytorch构建自己的深度学习模型的时候,基本上都是下面这个...
  • MLOps社区实验1:小组3:Yelp评论分类 请在我们的找到更多信息 :upside-down_face: 很棒的MLOps社区 :party_popper: 是面向MLOps的开放,... 请在./src中找到代码。 非常感谢@ paulomaia20处理此问题! :nerd_face:
  • 本文收集了大量基于 PyTorch 实现的代码链接,其中有适用于深度学习新手的“入门指导系列”,也有适用于老司机的论文代码实现,包括 Attention Based CNN、A3C、WGAN等等。所有代码均按照所属技术领域分类,包括机器...
  • Focal Loss 分类问题 pytorch实现代码(简单实现)

    千次阅读 热门讨论 2019-06-11 18:19:20
    首先感谢Code_Mart的博客把理论汇总了下https://blog.csdn.net/Code_Mart/article/details/89736187.并实现了Focal Loss的二分类和多分类的代码并讲解.同时他与xwmwanjy666的讨论更讲清楚了...
  • 2019论文的pytorch演示代码DuLa-Net:用于从单个RGB全景估算房间布局的双投影网络(,) 通过此存储库,您可以从单个室内RGB全景图估算3D房间布局。 要查看更多详细信息,请参阅论文或项目页面。 先决条件 Python3 ...
  • DDPG强化学习pytorch代码

    万次阅读 2019-01-10 14:48:27
    DDPG强化学习pytorch代码 参照莫烦大神的强化学习教程tensorflow代码改写成了pytorch代码。 具体代码如下. ''' torch = 0.41 ''' import torch import torch.nn as nn import torch.nn.functional as F ...
  • ps:感谢Code_Mart的解答,肯定了思路,不过他也不确定是否可以在pytorch中那么写.事情这样模棱两可让我很烦躁决定深究一下.看到博客https://blog.csdn.net/qq_22210253/article/details/85229988对CrossEntropy...
  • 在vs2010 MFC c++代码中需要调用pytorch写的深度学习的代码以获得预测结果; 具体实现方法为: vs中输出待预测数据,pytorch执行预测结果后写出数据,然后vs重新读取预测数据。 在c++中加入代码段:batpath 为...

空空如也

空空如也

1 2 3 4 5 ... 20
收藏数 18,753
精华内容 7,501
关键字:

pytorch预测代码