Borch

Borch is a universal probabilistic programing language built on top of pytorch and is used for probabilistic modeling.

It is designed for fast experimentation and research with probabilistic models. With key focus on beeing very flexible and expressive withouth sacreficing the usabiilty. Thus it can be used for a wide range of models from classical hierarchical models on small data sets to complex deep probabilistic models on large data sets.

borch separates model specification(borch) and inference(infer) into two seperate packages for maximaple flexability and usability. However borch provides a clean interface to use in combination with the `infer’.

Examples

>>> from borch.utils import torch_utils
>>> from borch import infer, Module, RandomVariable, sample, pq_to_infer
>>> from borch.optimizer_collection import OptimizersCollection
>>> import borch.distributions as dist
>>> import numpy as np
>>> import torch
>>> from torch import optim

Generate random data

>>> def generate_dataset(n=100):
...     x = np.linspace(0, 10, n)
...     y = 2*x+4+np.random.normal(0, 4, n)
...     return (torch.tensor(y, dtype=torch.float32),
... torch.tensor(x, dtype=torch.float32))
>>> y, x = generate_dataset(100)

Defining a linear regression model

>>> def forward(bm, x):
...     bm.b = dist.Normal(0, 3)
...     bm.a = dist.Normal(0, 3)
...     bm.sigma = dist.LogNormal(1, 1)
...     mu = bm.b * x + bm.a
...     bm.y = dist.Normal(mu, bm.sigma)
...     return bm.y
>>> model = Module()
>>> model.observe(y=y)
>>> optimizer = OptimizersCollection(optimizer=optim.Adam, lr=0.01,
...     amsgrad=True)

Training loop

>>> for e in range(2):
...     sample(model)
...     yhat = forward(model, x)
...     loss = infer.vi_loss(**pq_to_infer(model))
...     loss.backward()
...     optimizer.step(model.parameters())
...     optimizer.zero_grad()

class borch.Graph(posterior=None)

Bases: borch.module.Module

A borch.Module that can act as a tensor.

A graph``s forward takes no arguments and returns a single tensor. This tensor is stored with the graph and the graph iteself can act as the tensor.

Note: This is a base class and is intended to be inherited from and not be used directly.

Examples

>>> import torch
>>> class Exp(Graph):
...     'Apply the exp transform'
...     def __init__(self, param):
...         super().__init__()
...         self.register_param_or_buffer("param", param)
...     def forward(self):
...         return torch.exp(self.param)

>>> exp = Exp(torch.nn.Parameter(torch.zeros(1)))
>>> exp*1
tensor([1.], grad_fn=<MulBackward0>)
>>> list(exp.parameters())
[Parameter containing:
tensor([0.], requires_grad=True)]

property T

Is this Tensor with its dimensions reversed.

If n is the number of dimensions in x, x.T is equivalent to x.permute(n-1, n-2, ..., 0).

abs() → Tensor

See torch.abs()

abs_() → Tensor

In-place version of abs()

absolute() → Tensor

Alias for abs()

absolute_() → Tensor

In-place version of absolute() Alias for abs_()

acos() → Tensor

See torch.acos()

acos_() → Tensor

In-place version of acos()

acosh() → Tensor

See torch.acosh()

acosh_() → Tensor

In-place version of acosh()

add(other, *, alpha=1) → Tensor

Add a scalar or tensor to self tensor. If both alpha and other are specified, each element of other is scaled by alpha before being used.

When other is a tensor, the shape of other must be broadcastable with the shape of the underlying tensor

See torch.add()

add_(other, *, alpha=1) → Tensor

In-place version of add()

addbmm(batch1, batch2, *, beta=1, alpha=1) → Tensor

See torch.addbmm()

addbmm_(batch1, batch2, *, beta=1, alpha=1) → Tensor

In-place version of addbmm()

addcdiv(tensor1, tensor2, *, value=1) → Tensor

See torch.addcdiv()

addcdiv_(tensor1, tensor2, *, value=1) → Tensor

In-place version of addcdiv()

addcmul(tensor1, tensor2, *, value=1) → Tensor

See torch.addcmul()

addcmul_(tensor1, tensor2, *, value=1) → Tensor

In-place version of addcmul()

addmm(mat1, mat2, *, beta=1, alpha=1) → Tensor

See torch.addmm()

addmm_(mat1, mat2, *, beta=1, alpha=1) → Tensor

In-place version of addmm()

addmv(mat, vec, *, beta=1, alpha=1) → Tensor

See torch.addmv()

addmv_(mat, vec, *, beta=1, alpha=1) → Tensor

In-place version of addmv()

addr(vec1, vec2, *, beta=1, alpha=1) → Tensor

See torch.addr()

addr_(vec1, vec2, *, beta=1, alpha=1) → Tensor

In-place version of addr()

align_as(other) → Tensor

Permutes the dimensions of the self tensor to match the dimension order in the other tensor, adding size-one dims for any new names.

This operation is useful for explicit broadcasting by names (see examples).

All of the dims of self must be named in order to use this method. The resulting tensor is a view on the original tensor.

All dimension names of self must be present in other.names. other may contain named dimensions that are not in self.names; the output tensor has a size-one dimension for each of those new names.

To align a tensor to a specific order, use align_to().

Examples:

# Example 1: Applying a mask
>> mask = torch.randint(2, [127, 128], dtype=torch.bool).refine_names('W', 'H')
>> imgs = torch.randn(32, 128, 127, 3, names=('N', 'H', 'W', 'C'))
>> imgs.masked_fill_(mask.align_as(imgs), 0)


# Example 2: Applying a per-channel-scale
>> def scale_channels(input, scale):
>>    scale = scale.refine_names('C')
>>    return input * scale.align_as(input)

>> num_channels = 3
>> scale = torch.randn(num_channels, names=('C',))
>> imgs = torch.rand(32, 128, 128, num_channels, names=('N', 'H', 'W', 'C'))
>> more_imgs = torch.rand(32, num_channels, 128, 128, names=('N', 'C', 'H', 'W'))
>> videos = torch.randn(3, num_channels, 128, 128, 128, names=('N', 'C', 'H', 'W', 'D'))

# scale_channels is agnostic to the dimension order of the input
>> scale_channels(imgs, scale)
>> scale_channels(more_imgs, scale)
>> scale_channels(videos, scale)

Warning

The named tensor API is experimental and subject to change.

align_to(*names)

Permutes the dimensions of the self tensor to match the order specified in names, adding size-one dims for any new names.

All of the dims of self must be named in order to use this method. The resulting tensor is a view on the original tensor.

All dimension names of self must be present in names. names may contain additional names that are not in self.names; the output tensor has a size-one dimension for each of those new names.

names may contain up to one Ellipsis (...). The Ellipsis is expanded to be equal to all dimension names of self that are not mentioned in names, in the order that they appear in self.

Python 2 does not support Ellipsis but one may use a string literal instead ('...').

Parameters

names (iterable of str) – The desired dimension ordering of the output tensor. May contain up to one Ellipsis that is expanded to all unmentioned dim names of self.

Examples:

>> tensor = torch.randn(2, 2, 2, 2, 2, 2)
>> named_tensor = tensor.refine_names('A', 'B', 'C', 'D', 'E', 'F')

# Move the F and E dims to the front while keeping the rest in order
>> named_tensor.align_to('F', 'E', ...)

Warning

The named tensor API is experimental and subject to change.

all(dim=None, keepdim=False) → Tensor

See torch.all()

allclose(other, rtol=1e-05, atol=1e-08, equal_nan=False) → Tensor

See torch.allclose()

amax(dim=None, keepdim=False) → Tensor

See torch.amax()

amin(dim=None, keepdim=False) → Tensor

See torch.amin()

aminmax(*, dim=None, keepdim=False) -> (Tensor min, Tensor max)

See torch.aminmax()

angle() → Tensor

See torch.angle()

any(dim=None, keepdim=False) → Tensor

See torch.any()

apply_(callable) → Tensor

Applies the function callable to each element in the tensor, replacing each element with the value returned by callable.

Note

This function only works with CPU tensors and should not be used in code sections that require high performance.

arccos() → Tensor

See torch.arccos()

arccos_() → Tensor

In-place version of arccos()

arccosh()

acosh() -> Tensor

See torch.arccosh()

arccosh_()

acosh_() -> Tensor

In-place version of arccosh()

arcsin() → Tensor

See torch.arcsin()

arcsin_() → Tensor

In-place version of arcsin()

arcsinh() → Tensor

See torch.arcsinh()

arcsinh_() → Tensor

In-place version of arcsinh()

arctan() → Tensor

See torch.arctan()

arctan_() → Tensor

In-place version of arctan()

arctanh() → Tensor

See torch.arctanh()

arctanh_(other) → Tensor

In-place version of arctanh()

argmax(dim=None, keepdim=False) → LongTensor

See torch.argmax()

argmin(dim=None, keepdim=False) → LongTensor

See torch.argmin()

argsort(dim=-1, descending=False) → LongTensor

See torch.argsort()

as_strided(size, stride, storage_offset=0) → Tensor

See torch.as_strided()

as_subclass(cls) → Tensor

Makes a cls instance with the same data pointer as self. Changes in the output mirror changes in self, and the output stays attached to the autograd graph. cls must be a subclass of Tensor.

asin() → Tensor

See torch.asin()

asin_() → Tensor

In-place version of asin()

asinh() → Tensor

See torch.asinh()

asinh_() → Tensor

In-place version of asinh()

atan() → Tensor

See torch.atan()

atan2(other) → Tensor

See torch.atan2()

atan2_(other) → Tensor

In-place version of atan2()

atan_() → Tensor

In-place version of atan()

atanh() → Tensor

See torch.atanh()

atanh_(other) → Tensor

In-place version of atanh()

backward(gradient=None, retain_graph=None, create_graph=False, inputs=None)

Computes the gradient of current tensor w.r.t. graph leaves.

The graph is differentiated using the chain rule. If the tensor is non-scalar (i.e. its data has more than one element) and requires gradient, the function additionally requires specifying gradient. It should be a tensor of matching type and location, that contains the gradient of the differentiated function w.r.t. self.

This function accumulates gradients in the leaves - you might need to zero .grad attributes or set them to None before calling it. See Default gradient layouts for details on the memory layout of accumulated gradients.

Note

If you run any forward ops, create gradient, and/or call backward in a user-specified CUDA stream context, see Stream semantics of backward passes.

Note

When inputs are provided and a given input is not a leaf, the current implementation will call its grad_fn (though it is not strictly needed to get this gradients). It is an implementation detail on which the user should not rely. See https://github.com/pytorch/pytorch/pull/60521#issuecomment-867061780 for more details.

Parameters

• gradient (Tensor or None) – Gradient w.r.t. the tensor. If it is a tensor, it will be automatically converted to a Tensor that does not require grad unless create_graph is True. None values can be specified for scalar Tensors or ones that don’t require grad. If a None value would be acceptable then this argument is optional.

• retain_graph (bool, optional) – If False, the graph used to compute the grads will be freed. Note that in nearly all cases setting this option to True is not needed and often can be worked around in a much more efficient way. Defaults to the value of create_graph.

• create_graph (bool, optional) – If True, graph of the derivative will be constructed, allowing to compute higher order derivative products. Defaults to False.

• inputs (sequence of Tensor) – Inputs w.r.t. which the gradient will be accumulated into .grad. All other Tensors will be ignored. If not provided, the gradient is accumulated into all the leaf Tensors that were used to compute the attr::tensors.

baddbmm(batch1, batch2, *, beta=1, alpha=1) → Tensor

See torch.baddbmm()

baddbmm_(batch1, batch2, *, beta=1, alpha=1) → Tensor

In-place version of baddbmm()

bernoulli(*, generator=None) → Tensor

Returns a result tensor where each \(\texttt{result[i]}\) is independently sampled from \(\text{Bernoulli}(\texttt{self[i]})\). self must have floating point dtype, and the result will have the same dtype.

See torch.bernoulli()

bernoulli_(p=0.5, *, generator=None) → Tensor

Fills each location of self with an independent sample from \(\text{Bernoulli}(\texttt{p})\). self can have integral dtype.

p should either be a scalar or tensor containing probabilities to be used for drawing the binary random number.

If it is a tensor, the \(\text{i}^{th}\) element of self tensor will be set to a value sampled from \(\text{Bernoulli}(\texttt{p\_tensor[i]})\). In this case p must have floating point dtype.

See also bernoulli() and torch.bernoulli()

bincount(weights=None, minlength=0) → Tensor

See torch.bincount()

bitwise_and() → Tensor

See torch.bitwise_and()

bitwise_and_() → Tensor

In-place version of bitwise_and()

bitwise_left_shift(other) → Tensor

See torch.bitwise_left_shift()

bitwise_left_shift_(other) → Tensor

In-place version of bitwise_left_shift()

bitwise_not() → Tensor

See torch.bitwise_not()

bitwise_not_() → Tensor

In-place version of bitwise_not()

bitwise_or() → Tensor

See torch.bitwise_or()

bitwise_or_() → Tensor

In-place version of bitwise_or()

bitwise_right_shift(other) → Tensor

See torch.bitwise_right_shift()

bitwise_right_shift_(other) → Tensor

In-place version of bitwise_right_shift()

bitwise_xor() → Tensor

See torch.bitwise_xor()

bitwise_xor_() → Tensor

In-place version of bitwise_xor()

bmm(batch2) → Tensor

See torch.bmm()

bool(memory_format=torch.preserve_format) → Tensor

self.bool() is equivalent to self.to(torch.bool). See to().

Parameters

memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

broadcast_to(shape) → Tensor

See torch.broadcast_to().

byte(memory_format=torch.preserve_format) → Tensor

self.byte() is equivalent to self.to(torch.uint8). See to().

Parameters

memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

cauchy_(median=0, sigma=1, *, generator=None) → Tensor

Fills the tensor with numbers drawn from the Cauchy distribution:

\[f(x) = \dfrac{1}{\pi} \dfrac{\sigma}{(x - \text{median})^2 + \sigma^2}\]

cdouble(memory_format=torch.preserve_format) → Tensor

self.cdouble() is equivalent to self.to(torch.complex128). See to().

Parameters

memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

ceil() → Tensor

See torch.ceil()

ceil_() → Tensor

In-place version of ceil()

cfloat(memory_format=torch.preserve_format) → Tensor

self.cfloat() is equivalent to self.to(torch.complex64). See to().

Parameters

memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

char(memory_format=torch.preserve_format) → Tensor

self.char() is equivalent to self.to(torch.int8). See to().

Parameters

memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

cholesky(upper=False) → Tensor

See torch.cholesky()

cholesky_inverse(upper=False) → Tensor

See torch.cholesky_inverse()

cholesky_solve(input2, upper=False) → Tensor

See torch.cholesky_solve()

chunk(chunks, dim=0) → List of Tensors

See torch.chunk()

clamp(min=None, max=None) → Tensor

See torch.clamp()

clamp_(min=None, max=None) → Tensor

In-place version of clamp()

clip(min=None, max=None) → Tensor

Alias for clamp().

clip_(min=None, max=None) → Tensor

Alias for clamp_().

clone(*, memory_format=torch.preserve_format) → Tensor

See torch.clone()

coalesce() → Tensor

Returns a coalesced copy of self if self is an uncoalesced tensor.

Returns self if self is a coalesced tensor.

Warning

Throws an error if self is not a sparse COO tensor.

col_indices() → IntTensor

Returns the tensor containing the column indices of the self tensor when self is a sparse CSR tensor of layout sparse_csr. The col_indices tensor is strictly of shape (self.nnz()) and of type int32 or int64. When using MKL routines such as sparse matrix multiplication, it is necessary to use int32 indexing in order to avoid downcasting and potentially losing information.

Example::

>> csr = torch.eye(5,5).to_sparse_csr() >> csr.col_indices() tensor([0, 1, 2, 3, 4], dtype=torch.int32)

conj() → Tensor

See torch.conj()

conj_physical() → Tensor

See torch.conj_physical()

conj_physical_() → Tensor

In-place version of conj_physical()

contiguous(memory_format=torch.contiguous_format) → Tensor

Returns a contiguous in memory tensor containing the same data as self tensor. If self tensor is already in the specified memory format, this function returns the self tensor.

Parameters

memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.contiguous_format.

copy_(src, non_blocking=False) → Tensor

Copies the elements from src into self tensor and returns self.

The src tensor must be broadcastable with the self tensor. It may be of a different data type or reside on a different device.

Parameters

• src (Tensor) – the source tensor to copy from

• non_blocking (bool) – if True and this copy is between CPU and GPU, the copy may occur asynchronously with respect to the host. For other cases, this argument has no effect.

copysign(other) → Tensor

See torch.copysign()

copysign_(other) → Tensor

In-place version of copysign()

corrcoef() → Tensor

See torch.corrcoef()

cos() → Tensor

See torch.cos()

cos_() → Tensor

In-place version of cos()

cosh() → Tensor

See torch.cosh()

cosh_() → Tensor

In-place version of cosh()

count_nonzero(dim=None) → Tensor

See torch.count_nonzero()

cov(*, correction=1, fweights=None, aweights=None) → Tensor

See torch.cov()

cross(other, dim=-1) → Tensor

See torch.cross()

crow_indices() → IntTensor

Returns the tensor containing the compressed row indices of the self tensor when self is a sparse CSR tensor of layout sparse_csr. The crow_indices tensor is strictly of shape (self.size(0) + 1) and of type int32 or int64. When using MKL routines such as sparse matrix multiplication, it is necessary to use int32 indexing in order to avoid downcasting and potentially losing information.

Example::

>> csr = torch.eye(5,5).to_sparse_csr() >> csr.crow_indices() tensor([0, 1, 2, 3, 4, 5], dtype=torch.int32)

cummax(dim) -> (Tensor, Tensor)

See torch.cummax()

cummin(dim) -> (Tensor, Tensor)

See torch.cummin()

cumprod(dim, dtype=None) → Tensor

See torch.cumprod()

cumprod_(dim, dtype=None) → Tensor

In-place version of cumprod()

cumsum(dim, dtype=None) → Tensor

See torch.cumsum()

cumsum_(dim, dtype=None) → Tensor

In-place version of cumsum()

data_ptr() → int

Returns the address of the first element of self tensor.

deg2rad() → Tensor

See torch.deg2rad()

deg2rad_() → Tensor

In-place version of deg2rad()

dense_dim() → int

Return the number of dense dimensions in a sparse tensor self.

Warning

Throws an error if self is not a sparse tensor.

See also Tensor.sparse_dim() and hybrid tensors.

dequantize() → Tensor

Given a quantized Tensor, dequantize it and return the dequantized float Tensor.

det() → Tensor

See torch.det()

detach()

Returns a new Tensor, detached from the current graph.

The result will never require gradient.

This method also affects forward mode AD gradients and the result will never have forward mode AD gradients.

Note

Returned Tensor shares the same storage with the original one. In-place modifications on either of them will be seen, and may trigger errors in correctness checks. IMPORTANT NOTE: Previously, in-place size / stride / storage changes (such as resize_ / resize_as_ / set_ / transpose_) to the returned tensor also update the original tensor. Now, these in-place changes will not update the original tensor anymore, and will instead trigger an error. For sparse tensors: In-place indices / values changes (such as zero_ / copy_ / add_) to the returned tensor will not update the original tensor anymore, and will instead trigger an error.

detach_()

Detaches the Tensor from the graph that created it, making it a leaf. Views cannot be detached in-place.

This method also affects forward mode AD gradients and the result will never have forward mode AD gradients.

property device

Is the torch.device where this Tensor is.

diag(diagonal=0) → Tensor

See torch.diag()

diag_embed(offset=0, dim1=-2, dim2=-1) → Tensor

See torch.diag_embed()

diagflat(offset=0) → Tensor

See torch.diagflat()

diagonal(offset=0, dim1=0, dim2=1) → Tensor

See torch.diagonal()

diff(n=1, dim=-1, prepend=None, append=None) → Tensor

See torch.diff()

digamma() → Tensor

See torch.digamma()

digamma_() → Tensor

In-place version of digamma()

dim() → int

Returns the number of dimensions of self tensor.

dist(other, p=2) → Tensor

See torch.dist()

div(value, *, rounding_mode=None) → Tensor

See torch.div()

div_(value, *, rounding_mode=None) → Tensor

In-place version of div()

divide(value, *, rounding_mode=None) → Tensor

See torch.divide()

divide_(value, *, rounding_mode=None) → Tensor

In-place version of divide()

dot(other) → Tensor

See torch.dot()

dsplit(split_size_or_sections) → List of Tensors

See torch.dsplit()

eig(eigenvectors=False) -> (Tensor, Tensor)

See torch.eig()

element_size() → int

Returns the size in bytes of an individual element.

Example:

>> torch.tensor([]).element_size()
4
>> torch.tensor([], dtype=torch.uint8).element_size()
1

eq(other) → Tensor

See torch.eq()

eq_(other) → Tensor

In-place version of eq()

equal(other) → bool

See torch.equal()

erf() → Tensor

See torch.erf()

erf_() → Tensor

In-place version of erf()

erfc() → Tensor

See torch.erfc()

erfc_() → Tensor

In-place version of erfc()

erfinv() → Tensor

See torch.erfinv()

erfinv_() → Tensor

In-place version of erfinv()

exp() → Tensor

See torch.exp()

exp2() → Tensor

See torch.exp2()

exp2_() → Tensor

In-place version of exp2()

exp_() → Tensor

In-place version of exp()

expand(*sizes) → Tensor

Returns a new view of the self tensor with singleton dimensions expanded to a larger size.

Passing -1 as the size for a dimension means not changing the size of that dimension.

Tensor can be also expanded to a larger number of dimensions, and the new ones will be appended at the front. For the new dimensions, the size cannot be set to -1.

Expanding a tensor does not allocate new memory, but only creates a new view on the existing tensor where a dimension of size one is expanded to a larger size by setting the stride to 0. Any dimension of size 1 can be expanded to an arbitrary value without allocating new memory.

Parameters

*sizes (torch.Size or int...) – the desired expanded size

Warning

More than one element of an expanded tensor may refer to a single memory location. As a result, in-place operations (especially ones that are vectorized) may result in incorrect behavior. If you need to write to the tensors, please clone them first.

Example:

>> x = torch.tensor([[1], [2], [3]])
>> x.size()
torch.Size([3, 1])
>> x.expand(3, 4)
tensor([[ 1,  1,  1,  1],
        [ 2,  2,  2,  2],
        [ 3,  3,  3,  3]])
>> x.expand(-1, 4)   # -1 means not changing the size of that dimension
tensor([[ 1,  1,  1,  1],
        [ 2,  2,  2,  2],
        [ 3,  3,  3,  3]])

expand_as(other) → Tensor

Expand this tensor to the same size as other. self.expand_as(other) is equivalent to self.expand(other.size()).

Please see expand() for more information about expand.

Parameters

other (torch.Tensor) – The result tensor has the same size as other.

expm1() → Tensor

See torch.expm1()

expm1_() → Tensor

In-place version of expm1()

exponential_(lambd=1, *, generator=None) → Tensor

Fills self tensor with elements drawn from the exponential distribution:

\[f(x) = \lambda e^{-\lambda x}\]

fill_(value) → Tensor

Fills self tensor with the specified value.

fill_diagonal_(fill_value, wrap=False) → Tensor

Fill the main diagonal of a tensor that has at least 2-dimensions. When dims>2, all dimensions of input must be of equal length. This function modifies the input tensor in-place, and returns the input tensor.

Parameters

• fill_value (Scalar) – the fill value

• wrap (bool) – the diagonal ‘wrapped’ after N columns for tall matrices.

Example:

>> a = torch.zeros(3, 3)
>> a.fill_diagonal_(5)
tensor([[5., 0., 0.],
        [0., 5., 0.],
        [0., 0., 5.]])
>> b = torch.zeros(7, 3)
>> b.fill_diagonal_(5)
tensor([[5., 0., 0.],
        [0., 5., 0.],
        [0., 0., 5.],
        [0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.]])
>> c = torch.zeros(7, 3)
>> c.fill_diagonal_(5, wrap=True)
tensor([[5., 0., 0.],
        [0., 5., 0.],
        [0., 0., 5.],
        [0., 0., 0.],
        [5., 0., 0.],
        [0., 5., 0.],
        [0., 0., 5.]])

fix() → Tensor

See torch.fix().

fix_() → Tensor

In-place version of fix()

flatten(start_dim=0, end_dim=-1) → Tensor

See torch.flatten()

flip(dims) → Tensor

See torch.flip()

fliplr() → Tensor

See torch.fliplr()

flipud() → Tensor

See torch.flipud()

float_power(exponent) → Tensor

See torch.float_power()

float_power_(exponent) → Tensor

In-place version of float_power()

floor() → Tensor

See torch.floor()

floor_() → Tensor

In-place version of floor()

floor_divide(value) → Tensor

See torch.floor_divide()

floor_divide_(value) → Tensor

In-place version of floor_divide()

fmax(other) → Tensor

See torch.fmax()

fmin(other) → Tensor

See torch.fmin()

fmod(divisor) → Tensor

See torch.fmod()

fmod_(divisor) → Tensor

In-place version of fmod()

frac() → Tensor

See torch.frac()

frac_() → Tensor

In-place version of frac()

frexp(input) -> (Tensor mantissa, Tensor exponent)

See torch.frexp()

gather(dim, index) → Tensor

See torch.gather()

gcd(other) → Tensor

See torch.gcd()

gcd_(other) → Tensor

In-place version of gcd()

ge(other) → Tensor

See torch.ge().

ge_(other) → Tensor

In-place version of ge().

geometric_(p, *, generator=None) → Tensor

Fills self tensor with elements drawn from the geometric distribution:

\[f(X=k) = p^{k - 1} (1 - p)\]

geqrf() -> (Tensor, Tensor)

See torch.geqrf()

ger(vec2) → Tensor

See torch.ger()

get_device() -> Device ordinal (Integer)

For CUDA tensors, this function returns the device ordinal of the GPU on which the tensor resides. For CPU tensors, an error is thrown.

Example:

>> x = torch.randn(3, 4, 5, device='cuda:0')
>> x.get_device()
0
>> x.cpu().get_device()  # RuntimeError: get_device is not implemented for type torch.FloatTensor

property grad

This attribute is None by default and becomes a Tensor the first time a call to backward() computes gradients for self. The attribute will then contain the gradients computed and future calls to backward() will accumulate (add) gradients into it.

greater(other) → Tensor

See torch.greater().

greater_(other) → Tensor

In-place version of greater().

greater_equal(other) → Tensor

See torch.greater_equal().

greater_equal_(other) → Tensor

In-place version of greater_equal().

gt(other) → Tensor

See torch.gt().

gt_(other) → Tensor

In-place version of gt().

hardshrink(lambd=0.5) → Tensor

See torch.nn.functional.hardshrink()

has_been_run()

Check if the graph has been executed such there is a .tensor value

has_names()

Is True if any of this tensor’s dimensions are named. Otherwise, is False.

heaviside(values) → Tensor

See torch.heaviside()

heaviside_(values) → Tensor

In-place version of heaviside()

histc(bins=100, min=0, max=0) → Tensor

See torch.histc()

histogram(input, bins, *, range=None, weight=None, density=False) -> (Tensor, Tensor)

See torch.histogram()

hsplit(split_size_or_sections) → List of Tensors

See torch.hsplit()

hypot(other) → Tensor

See torch.hypot()

hypot_(other) → Tensor

In-place version of hypot()

i0() → Tensor

See torch.i0()

i0_() → Tensor

In-place version of i0()

igamma(other) → Tensor

See torch.igamma()

igamma_(other) → Tensor

In-place version of igamma()

igammac(other) → Tensor

See torch.igammac()

igammac_(other) → Tensor

In-place version of igammac()

index_add(dim, index, tensor2) → Tensor

Out-of-place version of torch.Tensor.index_add_().

index_add_(dim, index, tensor, *, alpha=1) → Tensor

Accumulate the elements of alpha times tensor into the self tensor by adding to the indices in the order given in index. For example, if dim == 0, index[i] == j, and alpha=-1, then the ith row of tensor is subtracted from the jth row of self.

The dimth dimension of tensor must have the same size as the length of index (which must be a vector), and all other dimensions must match self, or an error will be raised.

Note

This operation may behave nondeterministically when given tensors on a CUDA device. See /notes/randomness for more information.

Parameters

• dim (int) – dimension along which to index

• index (IntTensor or LongTensor) – indices of tensor to select from

• tensor (Tensor) – the tensor containing values to add

Keyword Arguments

alpha (Number) – the scalar multiplier for tensor

Example:

>> x = torch.ones(5, 3)
>> t = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=torch.float)
>> index = torch.tensor([0, 4, 2])
>> x.index_add_(0, index, t)
tensor([[  2.,   3.,   4.],
        [  1.,   1.,   1.],
        [  8.,   9.,  10.],
        [  1.,   1.,   1.],
        [  5.,   6.,   7.]])
>> x.index_add_(0, index, t, alpha=-1)
tensor([[  1.,   1.,   1.],
        [  1.,   1.,   1.],
        [  1.,   1.,   1.],
        [  1.,   1.,   1.],
        [  1.,   1.,   1.]])

index_copy(dim, index, tensor2) → Tensor

Out-of-place version of torch.Tensor.index_copy_().

index_copy_(dim, index, tensor) → Tensor

Copies the elements of tensor into the self tensor by selecting the indices in the order given in index. For example, if dim == 0 and index[i] == j, then the ith row of tensor is copied to the jth row of self.

The dimth dimension of tensor must have the same size as the length of index (which must be a vector), and all other dimensions must match self, or an error will be raised.

Note

If index contains duplicate entries, multiple elements from tensor will be copied to the same index of self. The result is nondeterministic since it depends on which copy occurs last.

Parameters

• dim (int) – dimension along which to index

• index (LongTensor) – indices of tensor to select from

• tensor (Tensor) – the tensor containing values to copy

Example:

>> x = torch.zeros(5, 3)
>> t = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=torch.float)
>> index = torch.tensor([0, 4, 2])
>> x.index_copy_(0, index, t)
tensor([[ 1.,  2.,  3.],
        [ 0.,  0.,  0.],
        [ 7.,  8.,  9.],
        [ 0.,  0.,  0.],
        [ 4.,  5.,  6.]])

index_fill(dim, index, value) → Tensor

Out-of-place version of torch.Tensor.index_fill_().

index_fill_(dim, index, value) → Tensor

Fills the elements of the self tensor with value value by selecting the indices in the order given in index.

Parameters

• dim (int) – dimension along which to index

• index (LongTensor) – indices of self tensor to fill in

• value (float) – the value to fill with

Example::

>> x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=torch.float) >> index = torch.tensor([0, 2]) >> x.index_fill_(1, index, -1) tensor([[-1., 2., -1.],

[-1., 5., -1.], [-1., 8., -1.]])

index_put(indices, values, accumulate=False) → Tensor

Out-place version of index_put_().

index_put_(indices, values, accumulate=False) → Tensor

Puts values from the tensor values into the tensor self using the indices specified in indices (which is a tuple of Tensors). The expression tensor.index_put_(indices, values) is equivalent to tensor[indices] = values. Returns self.

If accumulate is True, the elements in values are added to self. If accumulate is False, the behavior is undefined if indices contain duplicate elements.

Parameters

• indices (tuple of LongTensor) – tensors used to index into self.

• values (Tensor) – tensor of same dtype as self.

• accumulate (bool) – whether to accumulate into self

index_select(dim, index) → Tensor

See torch.index_select()

indices() → Tensor

Return the indices tensor of a sparse COO tensor.

Warning

Throws an error if self is not a sparse COO tensor.

See also Tensor.values().

Note

This method can only be called on a coalesced sparse tensor. See Tensor.coalesce() for details.

inner(other) → Tensor

See torch.inner().

int(memory_format=torch.preserve_format) → Tensor

self.int() is equivalent to self.to(torch.int32). See to().

Parameters

memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

int_repr() → Tensor

Given a quantized Tensor, self.int_repr() returns a CPU Tensor with uint8_t as data type that stores the underlying uint8_t values of the given Tensor.

inverse() → Tensor

See torch.inverse()

is_coalesced() → bool

Returns True if self is a sparse COO tensor that is coalesced, False otherwise.

Warning

Throws an error if self is not a sparse COO tensor.

See coalesce() and uncoalesced tensors.

is_complex() → bool

Returns True if the data type of self is a complex data type.

is_conj() → bool

Returns True if the conjugate bit of self is set to true.

is_contiguous(memory_format=torch.contiguous_format) → bool

Returns True if self tensor is contiguous in memory in the order specified by memory format.

Parameters

memory_format (torch.memory_format, optional) – Specifies memory allocation order. Default: torch.contiguous_format.

property is_cuda

Is True if the Tensor is stored on the GPU, False otherwise.

is_floating_point() → bool

Returns True if the data type of self is a floating point data type.

is_inference() → bool

See torch.is_inference()

property is_leaf

All Tensors that have requires_grad which is False will be leaf Tensors by convention.

For Tensors that have requires_grad which is True, they will be leaf Tensors if they were created by the user. This means that they are not the result of an operation and so grad_fn is None.

Only leaf Tensors will have their grad populated during a call to backward(). To get grad populated for non-leaf Tensors, you can use retain_grad().

Example:

>> a = torch.rand(10, requires_grad=True)
>> a.is_leaf
True
>> b = torch.rand(10, requires_grad=True).cuda()
>> b.is_leaf
False
# b was created by the operation that cast a cpu Tensor into a cuda Tensor
>> c = torch.rand(10, requires_grad=True) + 2
>> c.is_leaf
False
# c was created by the addition operation
>> d = torch.rand(10).cuda()
>> d.is_leaf
True
# d does not require gradients and so has no operation creating it (that is tracked by the autograd engine)
>> e = torch.rand(10).cuda().requires_grad_()
>> e.is_leaf
True
# e requires gradients and has no operations creating it
>> f = torch.rand(10, requires_grad=True, device="cuda")
>> f.is_leaf
True
# f requires grad, has no operation creating it

property is_meta

Is True if the Tensor is a meta tensor, False otherwise. Meta tensors are like normal tensors, but they carry no data.

is_neg() → bool

Returns True if the negative bit of self is set to true.

is_pinned()

Returns true if this tensor resides in pinned memory.

property is_quantized

Is True if the Tensor is quantized, False otherwise.

is_set_to(tensor) → bool

Returns True if both tensors are pointing to the exact same memory (same storage, offset, size and stride).

is_shared()

Checks if tensor is in shared memory.

This is always True for CUDA tensors.

is_signed() → bool

Returns True if the data type of self is a signed data type.

property is_sparse

Is True if the Tensor uses sparse storage layout, False otherwise.

property is_sparse_csr

Is True if the Tensor uses sparse CSR storage layout, False otherwise.

property is_xpu

Is True if the Tensor is stored on the XPU, False otherwise.

isclose(other, rtol=1e-05, atol=1e-08, equal_nan=False) → Tensor

See torch.isclose()

isfinite() → Tensor

See torch.isfinite()

isinf() → Tensor

See torch.isinf()

isnan() → Tensor

See torch.isnan()

isneginf() → Tensor

See torch.isneginf()

isposinf() → Tensor

See torch.isposinf()

isreal() → Tensor

See torch.isreal()

istft(n_fft: int, hop_length: Optional[int] = None, win_length: Optional[int] = None, window: Optional[torch.Tensor] = None, center: bool = True, normalized: bool = False, onesided: Optional[bool] = None, length: Optional[int] = None, return_complex: bool = False)

See torch.istft()

item() → number

Returns the value of this tensor as a standard Python number. This only works for tensors with one element. For other cases, see tolist().

This operation is not differentiable.

Example:

>> x = torch.tensor([1.0])
>> x.item()
1.0

kron(other) → Tensor

See torch.kron()

kthvalue(k, dim=None, keepdim=False) -> (Tensor, LongTensor)

See torch.kthvalue()

lcm(other) → Tensor

See torch.lcm()

lcm_(other) → Tensor

In-place version of lcm()

ldexp(other) → Tensor

See torch.ldexp()

ldexp_(other) → Tensor

In-place version of ldexp()

le(other) → Tensor

See torch.le().

le_(other) → Tensor

In-place version of le().

lerp(end, weight) → Tensor

See torch.lerp()

lerp_(end, weight) → Tensor

In-place version of lerp()

less()

lt(other) -> Tensor

See torch.less().

less_(other) → Tensor

In-place version of less().

less_equal(other) → Tensor

See torch.less_equal().

less_equal_(other) → Tensor

In-place version of less_equal().

lgamma() → Tensor

See torch.lgamma()

lgamma_() → Tensor

In-place version of lgamma()

log() → Tensor

See torch.log()

log10() → Tensor

See torch.log10()

log10_() → Tensor

In-place version of log10()

log1p() → Tensor

See torch.log1p()

log1p_() → Tensor

In-place version of log1p()

log2() → Tensor

See torch.log2()

log2_() → Tensor

In-place version of log2()

log_() → Tensor

In-place version of log()

log_normal_(mean=1, std=2, *, generator=None)

Fills self tensor with numbers samples from the log-normal distribution parameterized by the given mean \(\mu\) and standard deviation \(\sigma\). Note that mean and std are the mean and standard deviation of the underlying normal distribution, and not of the returned distribution:

\[f(x) = \dfrac{1}{x \sigma \sqrt{2\pi}}\ e^{-\frac{(\ln x - \mu)^2}{2\sigma^2}}\]

logaddexp(other) → Tensor

See torch.logaddexp()

logaddexp2(other) → Tensor

See torch.logaddexp2()

logcumsumexp(dim) → Tensor

See torch.logcumsumexp()

logdet() → Tensor

See torch.logdet()

logical_and() → Tensor

See torch.logical_and()

logical_and_() → Tensor

In-place version of logical_and()

logical_not() → Tensor

See torch.logical_not()

logical_not_() → Tensor

In-place version of logical_not()

logical_or() → Tensor

See torch.logical_or()

logical_or_() → Tensor

In-place version of logical_or()

logical_xor() → Tensor

See torch.logical_xor()

logical_xor_() → Tensor

In-place version of logical_xor()

logit() → Tensor

See torch.logit()

logit_() → Tensor

In-place version of logit()

logsumexp(dim, keepdim=False) → Tensor

See torch.logsumexp()

long(memory_format=torch.preserve_format) → Tensor

self.long() is equivalent to self.to(torch.int64). See to().

Parameters

memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

lstsq(A) -> (Tensor, Tensor)

See torch.lstsq()

lt(other) → Tensor

See torch.lt().

lt_(other) → Tensor

In-place version of lt().

lu(pivot=True, get_infos=False)

See torch.lu()

lu_solve(LU_data, LU_pivots) → Tensor

See torch.lu_solve()

map_(tensor, callable)

Applies callable for each element in self tensor and the given tensor and stores the results in self tensor. self tensor and the given tensor must be broadcastable.

The callable should have the signature:

def callable(a, b) -> number

masked_fill(mask, value) → Tensor

Out-of-place version of torch.Tensor.masked_fill_()

masked_fill_(mask, value)

Fills elements of self tensor with value where mask is True. The shape of mask must be broadcastable with the shape of the underlying tensor.

Parameters

• mask (BoolTensor) – the boolean mask

• value (float) – the value to fill in with

masked_scatter(mask, tensor) → Tensor

Out-of-place version of torch.Tensor.masked_scatter_()

masked_scatter_(mask, source)

Copies elements from source into self tensor at positions where the mask is True. The shape of mask must be broadcastable with the shape of the underlying tensor. The source should have at least as many elements as the number of ones in mask

Parameters

• mask (BoolTensor) – the boolean mask

• source (Tensor) – the tensor to copy from

Note

The mask operates on the self tensor, not on the given source tensor.

masked_select(mask) → Tensor

See torch.masked_select()

matmul(tensor2) → Tensor

See torch.matmul()

matrix_exp() → Tensor

See torch.matrix_exp()

matrix_power(n) → Tensor

Note

matrix_power() is deprecated, use torch.linalg.matrix_power() instead.

Alias for torch.linalg.matrix_power()

max(dim=None, keepdim=False) -> Tensor or (Tensor, Tensor)

See torch.max()

maximum(other) → Tensor

See torch.maximum()

mean(dim=None, keepdim=False, *, dtype=None) → Tensor

See torch.mean()

median(dim=None, keepdim=False) -> (Tensor, LongTensor)

See torch.median()

min(dim=None, keepdim=False) -> Tensor or (Tensor, Tensor)

See torch.min()

minimum(other) → Tensor

See torch.minimum()

mm(mat2) → Tensor

See torch.mm()

mode(dim=None, keepdim=False) -> (Tensor, LongTensor)

See torch.mode()

moveaxis(source, destination) → Tensor

See torch.moveaxis()

movedim(source, destination) → Tensor

See torch.movedim()

msort() → Tensor

See torch.msort()

mul(value) → Tensor

See torch.mul().

mul_(value) → Tensor

In-place version of mul().

multinomial(num_samples, replacement=False, *, generator=None) → Tensor

See torch.multinomial()

multiply(value) → Tensor

See torch.multiply().

multiply_(value) → Tensor

In-place version of multiply().

mv(vec) → Tensor

See torch.mv()

mvlgamma(p) → Tensor

See torch.mvlgamma()

mvlgamma_(p) → Tensor

In-place version of mvlgamma()

property names

Stores names for each of this tensor’s dimensions.

names[idx] corresponds to the name of tensor dimension idx. Names are either a string if the dimension is named or None if the dimension is unnamed.

Dimension names may contain characters or underscore. Furthermore, a dimension name must be a valid Python variable name (i.e., does not start with underscore).

Tensors may not have two named dimensions with the same name.

Warning

The named tensor API is experimental and subject to change.

nan_to_num(nan=0.0, posinf=None, neginf=None) → Tensor

See torch.nan_to_num().

nan_to_num_(nan=0.0, posinf=None, neginf=None) → Tensor

In-place version of nan_to_num().

nanmean(dim=None, keepdim=False, *, dtype=None) → Tensor

See torch.nanmean()

nanmedian(dim=None, keepdim=False) -> (Tensor, LongTensor)

See torch.nanmedian()

nanquantile(q, dim=None, keepdim=False) → Tensor

See torch.nanquantile()

nansum(dim=None, keepdim=False, dtype=None) → Tensor

See torch.nansum()

narrow(dimension, start, length) → Tensor

See torch.narrow()

Example:

>> x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
>> x.narrow(0, 0, 2)
tensor([[ 1,  2,  3],
        [ 4,  5,  6]])
>> x.narrow(1, 1, 2)
tensor([[ 2,  3],
        [ 5,  6],
        [ 8,  9]])

narrow_copy(dimension, start, length) → Tensor

Same as Tensor.narrow() except returning a copy rather than shared storage. This is primarily for sparse tensors, which do not have a shared-storage narrow method. Calling narrow_copy with dimemsion > self.sparse_dim() will return a copy with the relevant dense dimension narrowed, and self.shape updated accordingly.

property ndim

Alias for dim()

ndimension() → int

Alias for dim()

ne(other) → Tensor

See torch.ne().

ne_(other) → Tensor

In-place version of ne().

neg() → Tensor

See torch.neg()

neg_() → Tensor

In-place version of neg()

negative() → Tensor

See torch.negative()

negative_() → Tensor

In-place version of negative()

nelement() → int

Alias for numel()

new_empty(size, dtype=None, device=None, requires_grad=False) → Tensor

Returns a Tensor of size size filled with uninitialized data. By default, the returned Tensor has the same torch.dtype and torch.device as this tensor.

Parameters

• dtype (torch.dtype, optional) – the desired type of returned tensor. Default: if None, same torch.dtype as this tensor.

• device (torch.device, optional) – the desired device of returned tensor. Default: if None, same torch.device as this tensor.

• requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>> tensor = torch.ones(())
>> tensor.new_empty((2, 3))
tensor([[ 5.8182e-18,  4.5765e-41, -1.0545e+30],
        [ 3.0949e-41,  4.4842e-44,  0.0000e+00]])

new_empty_strided(size, stride, dtype=None, device=None, requires_grad=False) → Tensor

Returns a Tensor of size size and strides stride filled with uninitialized data. By default, the returned Tensor has the same torch.dtype and torch.device as this tensor.

Parameters

• dtype (torch.dtype, optional) – the desired type of returned tensor. Default: if None, same torch.dtype as this tensor.

• device (torch.device, optional) – the desired device of returned tensor. Default: if None, same torch.device as this tensor.

• requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>> tensor = torch.ones(())
>> tensor.new_empty_strided((2, 3), (3, 1))
tensor([[ 5.8182e-18,  4.5765e-41, -1.0545e+30],
        [ 3.0949e-41,  4.4842e-44,  0.0000e+00]])

new_full(size, fill_value, dtype=None, device=None, requires_grad=False) → Tensor

Returns a Tensor of size size filled with fill_value. By default, the returned Tensor has the same torch.dtype and torch.device as this tensor.

Parameters

• fill_value (scalar) – the number to fill the output tensor with.

• dtype (torch.dtype, optional) – the desired type of returned tensor. Default: if None, same torch.dtype as this tensor.

• device (torch.device, optional) – the desired device of returned tensor. Default: if None, same torch.device as this tensor.

• requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>> tensor = torch.ones((2,), dtype=torch.float64)
>> tensor.new_full((3, 4), 3.141592)
tensor([[ 3.1416,  3.1416,  3.1416,  3.1416],
        [ 3.1416,  3.1416,  3.1416,  3.1416],
        [ 3.1416,  3.1416,  3.1416,  3.1416]], dtype=torch.float64)

new_ones(size, dtype=None, device=None, requires_grad=False) → Tensor

Returns a Tensor of size size filled with 1. By default, the returned Tensor has the same torch.dtype and torch.device as this tensor.

Parameters

• size (int...) – a list, tuple, or torch.Size of integers defining the shape of the output tensor.

• dtype (torch.dtype, optional) – the desired type of returned tensor. Default: if None, same torch.dtype as this tensor.

• device (torch.device, optional) – the desired device of returned tensor. Default: if None, same torch.device as this tensor.

• requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>> tensor = torch.tensor((), dtype=torch.int32)
>> tensor.new_ones((2, 3))
tensor([[ 1,  1,  1],
        [ 1,  1,  1]], dtype=torch.int32)

new_tensor(data, dtype=None, device=None, requires_grad=False) → Tensor

Returns a new Tensor with data as the tensor data. By default, the returned Tensor has the same torch.dtype and torch.device as this tensor.

Warning

new_tensor() always copies data. If you have a Tensor data and want to avoid a copy, use torch.Tensor.requires_grad_() or torch.Tensor.detach(). If you have a numpy array and want to avoid a copy, use torch.from_numpy().

Warning

When data is a tensor x, new_tensor() reads out ‘the data’ from whatever it is passed, and constructs a leaf variable. Therefore tensor.new_tensor(x) is equivalent to x.clone().detach() and tensor.new_tensor(x, requires_grad=True) is equivalent to x.clone().detach().requires_grad_(True). The equivalents using clone() and detach() are recommended.

Parameters

• data (array_like) – The returned Tensor copies data.

• dtype (torch.dtype, optional) – the desired type of returned tensor. Default: if None, same torch.dtype as this tensor.

• device (torch.device, optional) – the desired device of returned tensor. Default: if None, same torch.device as this tensor.

• requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>> tensor = torch.ones((2,), dtype=torch.int8)
>> data = [[0, 1], [2, 3]]
>> tensor.new_tensor(data)
tensor([[ 0,  1],
        [ 2,  3]], dtype=torch.int8)

new_zeros(size, dtype=None, device=None, requires_grad=False) → Tensor

Returns a Tensor of size size filled with 0. By default, the returned Tensor has the same torch.dtype and torch.device as this tensor.

Parameters

• size (int...) – a list, tuple, or torch.Size of integers defining the shape of the output tensor.

• dtype (torch.dtype, optional) – the desired type of returned tensor. Default: if None, same torch.dtype as this tensor.

• device (torch.device, optional) – the desired device of returned tensor. Default: if None, same torch.device as this tensor.

• requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>> tensor = torch.tensor((), dtype=torch.float64)
>> tensor.new_zeros((2, 3))
tensor([[ 0.,  0.,  0.],
        [ 0.,  0.,  0.]], dtype=torch.float64)

nextafter(other) → Tensor

See torch.nextafter()

nextafter_(other) → Tensor

In-place version of nextafter()

nonzero() → LongTensor

See torch.nonzero()

norm(p='fro', dim=None, keepdim=False, dtype=None)

See torch.norm()

normal_(mean=0, std=1, *, generator=None) → Tensor

Fills self tensor with elements samples from the normal distribution parameterized by mean and std.

not_equal(other) → Tensor

See torch.not_equal().

not_equal_(other) → Tensor

In-place version of not_equal().

numel() → int

See torch.numel()

numpy() → numpy.ndarray

Returns self tensor as a NumPy ndarray. This tensor and the returned ndarray share the same underlying storage. Changes to self tensor will be reflected in the ndarray and vice versa.

orgqr(input2) → Tensor

See torch.orgqr()

ormqr(input2, input3, left=True, transpose=False) → Tensor

See torch.ormqr()

outer(vec2) → Tensor

See torch.outer().

permute(*dims) → Tensor

See torch.permute()

pin_memory() → Tensor

Copies the tensor to pinned memory, if it’s not already pinned.

pinverse() → Tensor

See torch.pinverse()

polygamma(n) → Tensor

See torch.polygamma()

polygamma_(n) → Tensor

In-place version of polygamma()

positive() → Tensor

See torch.positive()

pow(exponent) → Tensor

See torch.pow()

pow_(exponent) → Tensor

In-place version of pow()

prod(dim=None, keepdim=False, dtype=None) → Tensor

See torch.prod()

put(input, index, source, accumulate=False) → Tensor

Out-of-place version of torch.Tensor.put_(). input corresponds to self in torch.Tensor.put_().

put_(index, source, accumulate=False) → Tensor

Copies the elements from source into the positions specified by index. For the purpose of indexing, the self tensor is treated as if it were a 1-D tensor.

index and source need to have the same number of elements, but not necessarily the same shape.

If accumulate is True, the elements in source are added to self. If accumulate is False, the behavior is undefined if index contain duplicate elements.

Parameters

• index (LongTensor) – the indices into self

• source (Tensor) – the tensor containing values to copy from

• accumulate (bool) – whether to accumulate into self

Example:

>> src = torch.tensor([[4, 3, 5],
..                     [6, 7, 8]])
>> src.put_(torch.tensor([1, 3]), torch.tensor([9, 10]))
tensor([[  4,   9,   5],
        [ 10,   7,   8]])

q_per_channel_axis() → int

Given a Tensor quantized by linear (affine) per-channel quantization, returns the index of dimension on which per-channel quantization is applied.

q_per_channel_scales() → Tensor

Given a Tensor quantized by linear (affine) per-channel quantization, returns a Tensor of scales of the underlying quantizer. It has the number of elements that matches the corresponding dimensions (from q_per_channel_axis) of the tensor.

q_per_channel_zero_points() → Tensor

Given a Tensor quantized by linear (affine) per-channel quantization, returns a tensor of zero_points of the underlying quantizer. It has the number of elements that matches the corresponding dimensions (from q_per_channel_axis) of the tensor.

q_scale() → float

Given a Tensor quantized by linear(affine) quantization, returns the scale of the underlying quantizer().

q_zero_point() → int

Given a Tensor quantized by linear(affine) quantization, returns the zero_point of the underlying quantizer().

qr(some=True) -> (Tensor, Tensor)

See torch.qr()

qscheme() → torch.qscheme

Returns the quantization scheme of a given QTensor.

quantile(q, dim=None, keepdim=False) → Tensor

See torch.quantile()

rad2deg() → Tensor

See torch.rad2deg()

rad2deg_() → Tensor

In-place version of rad2deg()

random_(from=0, to=None, *, generator=None) → Tensor

Fills self tensor with numbers sampled from the discrete uniform distribution over [from, to - 1]. If not specified, the values are usually only bounded by self tensor’s data type. However, for floating point types, if unspecified, range will be [0, 2^mantissa] to ensure that every value is representable. For example, torch.tensor(1, dtype=torch.double).random_() will be uniform in [0, 2^53].

ravel(input) → Tensor

see torch.ravel()

reciprocal() → Tensor

See torch.reciprocal()

reciprocal_() → Tensor

In-place version of reciprocal()

record_stream(stream)

Ensures that the tensor memory is not reused for another tensor until all current work queued on stream are complete.

Note

The caching allocator is aware of only the stream where a tensor was allocated. Due to the awareness, it already correctly manages the life cycle of tensors on only one stream. But if a tensor is used on a stream different from the stream of origin, the allocator might reuse the memory unexpectedly. Calling this method lets the allocator know which streams have used the tensor.

refine_names(*names)

Refines the dimension names of self according to names.

Refining is a special case of renaming that “lifts” unnamed dimensions. A None dim can be refined to have any name; a named dim can only be refined to have the same name.

Because named tensors can coexist with unnamed tensors, refining names gives a nice way to write named-tensor-aware code that works with both named and unnamed tensors.

names may contain up to one Ellipsis (...). The Ellipsis is expanded greedily; it is expanded in-place to fill names to the same length as self.dim() using names from the corresponding indices of self.names.

Python 2 does not support Ellipsis but one may use a string literal instead ('...').

Parameters

names (iterable of str) – The desired names of the output tensor. May contain up to one Ellipsis.

Examples:

>> imgs = torch.randn(32, 3, 128, 128)
>> named_imgs = imgs.refine_names('N', 'C', 'H', 'W')
>> named_imgs.names
('N', 'C', 'H', 'W')

>> tensor = torch.randn(2, 3, 5, 7, 11)
>> tensor = tensor.refine_names('A', ..., 'B', 'C')
>> tensor.names
('A', None, None, 'B', 'C')

Warning

The named tensor API is experimental and subject to change.

register_hook(hook)

Registers a backward hook.

The hook will be called every time a gradient with respect to the Tensor is computed. The hook should have the following signature:

hook(grad) -> Tensor or None

The hook should not modify its argument, but it can optionally return a new gradient which will be used in place of grad.

This function returns a handle with a method handle.remove() that removes the hook from the module.

Example:

>> v = torch.tensor([0., 0., 0.], requires_grad=True)
>> h = v.register_hook(lambda grad: grad * 2)  # double the gradient
>> v.backward(torch.tensor([1., 2., 3.]))
>> v.grad

 2
 4
 6
[torch.FloatTensor of size (3,)]

>> h.remove()  # removes the hook

register_param_or_buffer(name, val)

Register val as a parameter if it is a Parameter as a tensor if it is or can be cast to a tensor and setattr as a fallback

remainder(divisor) → Tensor

See torch.remainder()

remainder_(divisor) → Tensor

In-place version of remainder()

rename(*names, **rename_map)

Renames dimension names of self.

There are two main usages:

self.rename(**rename_map) returns a view on tensor that has dims renamed as specified in the mapping rename_map.

self.rename(*names) returns a view on tensor, renaming all dimensions positionally using names. Use self.rename(None) to drop names on a tensor.

One cannot specify both positional args names and keyword args rename_map.

Examples:

>> imgs = torch.rand(2, 3, 5, 7, names=('N', 'C', 'H', 'W'))
>> renamed_imgs = imgs.rename(N='batch', C='channels')
>> renamed_imgs.names
('batch', 'channels', 'H', 'W')

>> renamed_imgs = imgs.rename(None)
>> renamed_imgs.names
(None,)

>> renamed_imgs = imgs.rename('batch', 'channel', 'height', 'width')
>> renamed_imgs.names
('batch', 'channel', 'height', 'width')

Warning

The named tensor API is experimental and subject to change.

rename_(*names, **rename_map)

In-place version of rename().

renorm(p, dim, maxnorm) → Tensor

See torch.renorm()

renorm_(p, dim, maxnorm) → Tensor

In-place version of renorm()

repeat(*sizes) → Tensor

Repeats this tensor along the specified dimensions.

Unlike expand(), this function copies the tensor’s data.

Warning

repeat() behaves differently from numpy.repeat, but is more similar to numpy.tile. For the operator similar to numpy.repeat, see torch.repeat_interleave().

Parameters

sizes (torch.Size or int...) – The number of times to repeat this tensor along each dimension

Example:

>> x = torch.tensor([1, 2, 3])
>> x.repeat(4, 2)
tensor([[ 1,  2,  3,  1,  2,  3],
        [ 1,  2,  3,  1,  2,  3],
        [ 1,  2,  3,  1,  2,  3],
        [ 1,  2,  3,  1,  2,  3]])
>> x.repeat(4, 2, 1).size()
torch.Size([4, 2, 3])

repeat_interleave(repeats, dim=None, *, output_size=None) → Tensor

See torch.repeat_interleave().

property requires_grad

Is True if gradients need to be computed for this Tensor, False otherwise.

Note

The fact that gradients need to be computed for a Tensor do not mean that the grad attribute will be populated, see is_leaf for more details.

reshape(*shape) → Tensor

Returns a tensor with the same data and number of elements as self but with the specified shape. This method returns a view if shape is compatible with the current shape. See torch.Tensor.view() on when it is possible to return a view.

See torch.reshape()

Parameters

shape (tuple of ints or int...) – the desired shape

reshape_as(other) → Tensor

Returns this tensor as the same shape as other. self.reshape_as(other) is equivalent to self.reshape(other.sizes()). This method returns a view if other.sizes() is compatible with the current shape. See torch.Tensor.view() on when it is possible to return a view.

Please see reshape() for more information about reshape.

Parameters

other (torch.Tensor) – The result tensor has the same shape as other.

resize_(*sizes, memory_format=torch.contiguous_format) → Tensor

Resizes self tensor to the specified size. If the number of elements is larger than the current storage size, then the underlying storage is resized to fit the new number of elements. If the number of elements is smaller, the underlying storage is not changed. Existing elements are preserved but any new memory is uninitialized.

Warning

This is a low-level method. The storage is reinterpreted as C-contiguous, ignoring the current strides (unless the target size equals the current size, in which case the tensor is left unchanged). For most purposes, you will instead want to use view(), which checks for contiguity, or reshape(), which copies data if needed. To change the size in-place with custom strides, see set_().

Parameters

• sizes (torch.Size or int...) – the desired size

• memory_format (torch.memory_format, optional) – the desired memory format of Tensor. Default: torch.contiguous_format. Note that memory format of self is going to be unaffected if self.size() matches sizes.

Example:

>> x = torch.tensor([[1, 2], [3, 4], [5, 6]])
>> x.resize_(2, 2)
tensor([[ 1,  2],
        [ 3,  4]])

resize_as_(tensor, memory_format=torch.contiguous_format) → Tensor

Resizes the self tensor to be the same size as the specified tensor. This is equivalent to self.resize_(tensor.size()).

Parameters

memory_format (torch.memory_format, optional) – the desired memory format of Tensor. Default: torch.contiguous_format. Note that memory format of self is going to be unaffected if self.size() matches tensor.size().

resolve_conj() → Tensor

See torch.resolve_conj()

resolve_neg() → Tensor

See torch.resolve_neg()

retain_grad() → None

Enables this Tensor to have their grad populated during backward(). This is a no-op for leaf tensors.

property retains_grad

Is True if this Tensor is non-leaf and its grad is enabled to be populated during backward(), False otherwise.

roll(shifts, dims) → Tensor

See torch.roll()

rot90(k, dims) → Tensor

See torch.rot90()

round() → Tensor

See torch.round()

round_() → Tensor

In-place version of round()

rsqrt() → Tensor

See torch.rsqrt()

rsqrt_() → Tensor

In-place version of rsqrt()

scatter(dim, index, src) → Tensor

Out-of-place version of torch.Tensor.scatter_()

scatter_(dim, index, src, reduce=None) → Tensor

Writes all values from the tensor src into self at the indices specified in the index tensor. For each value in src, its output index is specified by its index in src for dimension != dim and by the corresponding value in index for dimension = dim.

For a 3-D tensor, self is updated as:

self[index[i][j][k]][j][k] = src[i][j][k]  # if dim == 0
self[i][index[i][j][k]][k] = src[i][j][k]  # if dim == 1
self[i][j][index[i][j][k]] = src[i][j][k]  # if dim == 2

This is the reverse operation of the manner described in gather().

self, index and src (if it is a Tensor) should all have the same number of dimensions. It is also required that index.size(d) <= src.size(d) for all dimensions d, and that index.size(d) <= self.size(d) for all dimensions d != dim. Note that index and src do not broadcast.

Moreover, as for gather(), the values of index must be between 0 and self.size(dim) - 1 inclusive.

Warning

When indices are not unique, the behavior is non-deterministic (one of the values from src will be picked arbitrarily) and the gradient will be incorrect (it will be propagated to all locations in the source that correspond to the same index)!

Note

The backward pass is implemented only for src.shape == index.shape.

Additionally accepts an optional reduce argument that allows specification of an optional reduction operation, which is applied to all values in the tensor src into self at the indicies specified in the index. For each value in src, the reduction operation is applied to an index in self which is specified by its index in src for dimension != dim and by the corresponding value in index for dimension = dim.

Given a 3-D tensor and reduction using the multiplication operation, self is updated as:

self[index[i][j][k]][j][k] *= src[i][j][k]  # if dim == 0
self[i][index[i][j][k]][k] *= src[i][j][k]  # if dim == 1
self[i][j][index[i][j][k]] *= src[i][j][k]  # if dim == 2

Reducing with the addition operation is the same as using scatter_add_().

Parameters

• dim (int) – the axis along which to index

• index (LongTensor) – the indices of elements to scatter, can be either empty or of the same dimensionality as src. When empty, the operation returns self unchanged.

• src (Tensor or float) – the source element(s) to scatter.

• reduce (str, optional) – reduction operation to apply, can be either 'add' or 'multiply'.

Example:

>> src = torch.arange(1, 11).reshape((2, 5))
>> src
tensor([[ 1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10]])
>> index = torch.tensor([[0, 1, 2, 0]])
>> torch.zeros(3, 5, dtype=src.dtype).scatter_(0, index, src)
tensor([[1, 0, 0, 4, 0],
        [0, 2, 0, 0, 0],
        [0, 0, 3, 0, 0]])
>> index = torch.tensor([[0, 1, 2], [0, 1, 4]])
>> torch.zeros(3, 5, dtype=src.dtype).scatter_(1, index, src)
tensor([[1, 2, 3, 0, 0],
        [6, 7, 0, 0, 8],
        [0, 0, 0, 0, 0]])

>> torch.full((2, 4), 2.).scatter_(1, torch.tensor([[2], [3]]),
..            1.23, reduce='multiply')
tensor([[2.0000, 2.0000, 2.4600, 2.0000],
        [2.0000, 2.0000, 2.0000, 2.4600]])
>> torch.full((2, 4), 2.).scatter_(1, torch.tensor([[2], [3]]),
..            1.23, reduce='add')
tensor([[2.0000, 2.0000, 3.2300, 2.0000],
        [2.0000, 2.0000, 2.0000, 3.2300]])

scatter_add(dim, index, src) → Tensor

Out-of-place version of torch.Tensor.scatter_add_()

scatter_add_(dim, index, src) → Tensor

Adds all values from the tensor other into self at the indices specified in the index tensor in a similar fashion as scatter_(). For each value in src, it is added to an index in self which is specified by its index in src for dimension != dim and by the corresponding value in index for dimension = dim.

For a 3-D tensor, self is updated as:

self[index[i][j][k]][j][k] += src[i][j][k]  # if dim == 0
self[i][index[i][j][k]][k] += src[i][j][k]  # if dim == 1
self[i][j][index[i][j][k]] += src[i][j][k]  # if dim == 2

self, index and src should have same number of dimensions. It is also required that index.size(d) <= src.size(d) for all dimensions d, and that index.size(d) <= self.size(d) for all dimensions d != dim. Note that index and src do not broadcast.

Note

This operation may behave nondeterministically when given tensors on a CUDA device. See /notes/randomness for more information.

Note

The backward pass is implemented only for src.shape == index.shape.

Parameters

• dim (int) – the axis along which to index

• index (LongTensor) – the indices of elements to scatter and add, can be either empty or of the same dimensionality as src. When empty, the operation returns self unchanged.

• src (Tensor) – the source elements to scatter and add

Example:

>> src = torch.ones((2, 5))
>> index = torch.tensor([[0, 1, 2, 0, 0]])
>> torch.zeros(3, 5, dtype=src.dtype).scatter_add_(0, index, src)
tensor([[1., 0., 0., 1., 1.],
        [0., 1., 0., 0., 0.],
        [0., 0., 1., 0., 0.]])
>> index = torch.tensor([[0, 1, 2, 0, 0], [0, 1, 2, 2, 2]])
>> torch.zeros(3, 5, dtype=src.dtype).scatter_add_(0, index, src)
tensor([[2., 0., 0., 1., 1.],
        [0., 2., 0., 0., 0.],
        [0., 0., 2., 1., 1.]])

select(dim, index) → Tensor

Slices the self tensor along the selected dimension at the given index. This function returns a view of the original tensor with the given dimension removed.

Parameters

• dim (int) – the dimension to slice

• index (int) – the index to select with

Note

select() is equivalent to slicing. For example, tensor.select(0, index) is equivalent to tensor[index] and tensor.select(2, index) is equivalent to tensor[:,:,index].

set_(source=None, storage_offset=0, size=None, stride=None) → Tensor

Sets the underlying storage, size, and strides. If source is a tensor, self tensor will share the same storage and have the same size and strides as source. Changes to elements in one tensor will be reflected in the other.

If source is a Storage, the method sets the underlying storage, offset, size, and stride.

Parameters

• source (Tensor or Storage) – the tensor or storage to use

• storage_offset (int, optional) – the offset in the storage

• size (torch.Size, optional) – the desired size. Defaults to the size of the source.

• stride (tuple, optional) – the desired stride. Defaults to C-contiguous strides.

sgn() → Tensor

See torch.sgn()

sgn_() → Tensor

In-place version of sgn()

share_memory_()

Moves the underlying storage to shared memory.

This is a no-op if the underlying storage is already in shared memory and for CUDA tensors. Tensors in shared memory cannot be resized.

short(memory_format=torch.preserve_format) → Tensor

self.short() is equivalent to self.to(torch.int16). See to().

Parameters

memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

sigmoid() → Tensor

See torch.sigmoid()

sigmoid_() → Tensor

In-place version of sigmoid()

sign() → Tensor

See torch.sign()

sign_() → Tensor

In-place version of sign()

signbit() → Tensor

See torch.signbit()

sin() → Tensor

See torch.sin()

sin_() → Tensor

In-place version of sin()

sinc() → Tensor

See torch.sinc()

sinc_() → Tensor

In-place version of sinc()

sinh() → Tensor

See torch.sinh()

sinh_() → Tensor

In-place version of sinh()

size(dim=None) → torch.Size or int

Returns the size of the self tensor. If dim is not specified, the returned value is a torch.Size, a subclass of tuple. If dim is specified, returns an int holding the size of that dimension.

Parameters

dim (int, optional) – The dimension for which to retrieve the size.

Example:

>> t = torch.empty(3, 4, 5)
>> t.size()
torch.Size([3, 4, 5])
>> t.size(dim=1)
4

slogdet() -> (Tensor, Tensor)

See torch.slogdet()

smm(mat) → Tensor

See torch.smm()

solve(A) → Tensor, Tensor

See torch.solve()

sort(dim=-1, descending=False) -> (Tensor, LongTensor)

See torch.sort()

sparse_dim() → int

Return the number of sparse dimensions in a sparse tensor self.

Warning

Throws an error if self is not a sparse tensor.

See also Tensor.dense_dim() and hybrid tensors.

sparse_mask(mask) → Tensor

Returns a new sparse tensor with values from a strided tensor self filtered by the indices of the sparse tensor mask. The values of mask sparse tensor are ignored. self and mask tensors must have the same shape.

Note

The returned sparse tensor has the same indices as the sparse tensor mask, even when the corresponding values in self are zeros.

Parameters

mask (Tensor) – a sparse tensor whose indices are used as a filter

Example:

>> nse = 5
>> dims = (5, 5, 2, 2)
>> I = torch.cat([torch.randint(0, dims[0], size=(nse,)),
..                torch.randint(0, dims[1], size=(nse,))], 0).reshape(2, nse)
>> V = torch.randn(nse, dims[2], dims[3])
>> S = torch.sparse_coo_tensor(I, V, dims).coalesce()
>> D = torch.randn(dims)
>> D.sparse_mask(S)
tensor(indices=tensor([[0, 0, 0, 2],
                       [0, 1, 4, 3]]),
       values=tensor([[[ 1.6550,  0.2397],
                       [-0.1611, -0.0779]],

                      [[ 0.2326, -1.0558],
                       [ 1.4711,  1.9678]],

                      [[-0.5138, -0.0411],
                       [ 1.9417,  0.5158]],

                      [[ 0.0793,  0.0036],
                       [-0.2569, -0.1055]]]),
       size=(5, 5, 2, 2), nnz=4, layout=torch.sparse_coo)

sparse_resize_(size, sparse_dim, dense_dim) → Tensor

Resizes self sparse tensor to the desired size and the number of sparse and dense dimensions.

Note

If the number of specified elements in self is zero, then size, sparse_dim, and dense_dim can be any size and positive integers such that len(size) == sparse_dim + dense_dim.

If self specifies one or more elements, however, then each dimension in size must not be smaller than the corresponding dimension of self, sparse_dim must equal the number of sparse dimensions in self, and dense_dim must equal the number of dense dimensions in self.

Warning

Throws an error if self is not a sparse tensor.

Parameters

• size (torch.Size) – the desired size. If self is non-empty sparse tensor, the desired size cannot be smaller than the original size.

• sparse_dim (int) – the number of sparse dimensions

• dense_dim (int) – the number of dense dimensions

sparse_resize_and_clear_(size, sparse_dim, dense_dim) → Tensor

Removes all specified elements from a sparse tensor self and resizes self to the desired size and the number of sparse and dense dimensions.

Parameters

• size (torch.Size) – the desired size.

• sparse_dim (int) – the number of sparse dimensions

• dense_dim (int) – the number of dense dimensions

split(split_size, dim=0)

See torch.split()

sqrt() → Tensor

See torch.sqrt()

sqrt_() → Tensor

In-place version of sqrt()

square() → Tensor

See torch.square()

square_() → Tensor

In-place version of square()

squeeze(dim=None) → Tensor

See torch.squeeze()

squeeze_(dim=None) → Tensor

In-place version of squeeze()

sspaddmm(mat1, mat2, *, beta=1, alpha=1) → Tensor

See torch.sspaddmm()

std(dim, unbiased=True, keepdim=False) → Tensor

See torch.std()

std(unbiased=True) → Tensor

See torch.std()

stft(n_fft: int, hop_length: Optional[int] = None, win_length: Optional[int] = None, window: Optional[torch.Tensor] = None, center: bool = True, pad_mode: str = 'reflect', normalized: bool = False, onesided: Optional[bool] = None, return_complex: Optional[bool] = None)

See torch.stft()

Warning

This function changed signature at version 0.4.1. Calling with the previous signature may cause error or return incorrect result.

storage() → torch.Storage

Returns the underlying storage.

storage_offset() → int

Returns self tensor’s offset in the underlying storage in terms of number of storage elements (not bytes).

Example:

>> x = torch.tensor([1, 2, 3, 4, 5])
>> x.storage_offset()
0
>> x[3:].storage_offset()
3

storage_type() → type

Returns the type of the underlying storage.

stride(dim) → tuple or int

Returns the stride of self tensor.

Stride is the jump necessary to go from one element to the next one in the specified dimension dim. A tuple of all strides is returned when no argument is passed in. Otherwise, an integer value is returned as the stride in the particular dimension dim.

Parameters

dim (int, optional) – the desired dimension in which stride is required

Example:

>> x = torch.tensor([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]])
>> x.stride()
(5, 1)
>> x.stride(0)
5
>> x.stride(-1)
1

sub(other, *, alpha=1) → Tensor

See torch.sub().

sub_(other, *, alpha=1) → Tensor

In-place version of sub()

subtract(other, *, alpha=1) → Tensor

See torch.subtract().

subtract_(other, *, alpha=1) → Tensor

In-place version of subtract().

sum(dim=None, keepdim=False, dtype=None) → Tensor

See torch.sum()

sum_to_size(*size) → Tensor

Sum this tensor to size. size must be broadcastable to this tensor size.

Parameters

size (int...) – a sequence of integers defining the shape of the output tensor.

svd(some=True, compute_uv=True) -> (Tensor, Tensor, Tensor)

See torch.svd()

swapaxes(axis0, axis1) → Tensor

See torch.swapaxes()

swapaxes_(axis0, axis1) → Tensor

In-place version of swapaxes()

swapdims(dim0, dim1) → Tensor

See torch.swapdims()

swapdims_(dim0, dim1) → Tensor

In-place version of swapdims()

symeig(eigenvectors=False, upper=True) -> (Tensor, Tensor)

See torch.symeig()

t() → Tensor

See torch.t()

t_() → Tensor

In-place version of t()

take(indices) → Tensor

See torch.take()

take_along_dim(indices, dim) → Tensor

See torch.take_along_dim()

tan() → Tensor

See torch.tan()

tan_() → Tensor

In-place version of tan()

tanh() → Tensor

See torch.tanh()

tanh_() → Tensor

In-place version of tanh()

property tensor

Access the value of the last forward of the graph

tensor_split(indices_or_sections, dim=0) → List of Tensors

See torch.tensor_split()

tile(*reps) → Tensor

See torch.tile()

to_dense() → Tensor

Creates a strided copy of self.

Warning

Throws an error if self is a strided tensor.

Example:

>> s = torch.sparse_coo_tensor(
..        torch.tensor([[1, 1],
..                      [0, 2]]),
..        torch.tensor([9, 10]),
..        size=(3, 3))
>> s.to_dense()
tensor([[ 0,  0,  0],
        [ 9,  0, 10],
        [ 0,  0,  0]])

to_mkldnn() → Tensor

Returns a copy of the tensor in torch.mkldnn layout.

to_sparse(sparseDims) → Tensor

Returns a sparse copy of the tensor. PyTorch supports sparse tensors in coordinate format.

Parameters

sparseDims (int, optional) – the number of sparse dimensions to include in the new sparse tensor

Example:

>> d = torch.tensor([[0, 0, 0], [9, 0, 10], [0, 0, 0]])
>> d
tensor([[ 0,  0,  0],
        [ 9,  0, 10],
        [ 0,  0,  0]])
>> d.to_sparse()
tensor(indices=tensor([[1, 1],
                       [0, 2]]),
       values=tensor([ 9, 10]),
       size=(3, 3), nnz=2, layout=torch.sparse_coo)
>> d.to_sparse(1)
tensor(indices=tensor([[1]]),
       values=tensor([[ 9,  0, 10]]),
       size=(3, 3), nnz=1, layout=torch.sparse_coo)

to_sparse_csr()

Convert a tensor to compressed row storage format. Only works with 2D tensors.

Examples:

>> dense = torch.randn(5, 5)
>> sparse = dense.to_sparse_csr()
>> sparse._nnz()
25

tolist() → list or number

Returns the tensor as a (nested) list. For scalars, a standard Python number is returned, just like with item(). Tensors are automatically moved to the CPU first if necessary.

This operation is not differentiable.

Examples:

>> a = torch.randn(2, 2)
>> a.tolist()
[[0.012766935862600803, 0.5415473580360413],
 [-0.08909505605697632, 0.7729271650314331]]
>> a[0,0].tolist()
0.012766935862600803

topk(k, dim=None, largest=True, sorted=True) -> (Tensor, LongTensor)

See torch.topk()

trace() → Tensor

See torch.trace()

transpose(dim0, dim1) → Tensor

See torch.transpose()

transpose_(dim0, dim1) → Tensor

In-place version of transpose()

triangular_solve(A, upper=True, transpose=False, unitriangular=False) -> (Tensor, Tensor)

See torch.triangular_solve()

tril(k=0) → Tensor

See torch.tril()

tril_(k=0) → Tensor

In-place version of tril()

triu(k=0) → Tensor

See torch.triu()

triu_(k=0) → Tensor

In-place version of triu()

true_divide(value) → Tensor

See torch.true_divide()

true_divide_(value) → Tensor

In-place version of true_divide_()

trunc() → Tensor

See torch.trunc()

trunc_() → Tensor

In-place version of trunc()

type_as(tensor) → Tensor

Returns this tensor cast to the type of the given tensor.

This is a no-op if the tensor is already of the correct type. This is equivalent to self.type(tensor.type())

Parameters

tensor (Tensor) – the tensor which has the desired type

unbind(dim=0) → seq

See torch.unbind()

unflatten(dim, sizes)

Expands the dimension dim of the self tensor over multiple dimensions of sizes given by sizes.

• sizes is the new shape of the unflattened dimension and it can be a Tuple[int] as well as torch.Size if self is a Tensor, or namedshape (Tuple[(name: str, size: int)]) if self is a NamedTensor. The total number of elements in sizes must match the number of elements in the original dim being unflattened.

Parameters

• dim (Union[int, str]) – Dimension to unflatten

• sizes (Union[Tuple[int] or torch.Size, Tuple[Tuple[str, int]]]) – New shape of the unflattened dimension

Examples

>>>
>> torch.randn(3, 4, 1).unflatten(1, (2, 2)).shape
torch.Size([3, 2, 2, 1])
>> torch.randn(3, 4, 1).unflatten(1, (-1, 2)).shape # the size -1 is inferred from the size of dimension 1
torch.Size([3, 2, 2, 1])
>> torch.randn(2, 4, names=('A', 'B')).unflatten('B', (('B1', 2), ('B2', 2)))
tensor([[[-1.1772,  0.0180],
        [ 0.2412,  0.1431]],
        [[-1.1819, -0.8899],
        [ 1.5813,  0.2274]]], names=('A', 'B1', 'B2'))
>> torch.randn(2, names=('A',)).unflatten('A', (('B1', -1), ('B2', 1)))
tensor([[-0.8591],
        [ 0.3100]], names=('B1', 'B2'))

Warning

The named tensor API is experimental and subject to change.

unfold(dimension, size, step) → Tensor

Returns a view of the original tensor which contains all slices of size size from self tensor in the dimension dimension.

Step between two slices is given by step.

If sizedim is the size of dimension dimension for self, the size of dimension dimension in the returned tensor will be (sizedim - size) / step + 1.

An additional dimension of size size is appended in the returned tensor.

Parameters

• dimension (int) – dimension in which unfolding happens

• size (int) – the size of each slice that is unfolded

• step (int) – the step between each slice

Example:

>> x = torch.arange(1., 8)
>> x
tensor([ 1.,  2.,  3.,  4.,  5.,  6.,  7.])
>> x.unfold(0, 2, 1)
tensor([[ 1.,  2.],
        [ 2.,  3.],
        [ 3.,  4.],
        [ 4.,  5.],
        [ 5.,  6.],
        [ 6.,  7.]])
>> x.unfold(0, 2, 2)
tensor([[ 1.,  2.],
        [ 3.,  4.],
        [ 5.,  6.]])

uniform_(from=0, to=1) → Tensor

Fills self tensor with numbers sampled from the continuous uniform distribution:

\[P(x) = \dfrac{1}{\text{to} - \text{from}}\]

unique(sorted=True, return_inverse=False, return_counts=False, dim=None)

Returns the unique elements of the input tensor.

See torch.unique()

unique_consecutive(return_inverse=False, return_counts=False, dim=None)

Eliminates all but the first element from every consecutive group of equivalent elements.

See torch.unique_consecutive()

unsafe_chunk(chunks, dim=0) → List of Tensors

See torch.unsafe_chunk()

unsafe_split(split_size, dim=0) → List of Tensors

See torch.unsafe_split()

unsqueeze(dim) → Tensor

See torch.unsqueeze()

unsqueeze_(dim) → Tensor

In-place version of unsqueeze()

values() → Tensor

Return the values tensor of a sparse COO tensor.

Warning

Throws an error if self is not a sparse COO tensor.

See also Tensor.indices().

Note

This method can only be called on a coalesced sparse tensor. See Tensor.coalesce() for details.

var(dim, unbiased=True, keepdim=False) → Tensor

See torch.var()

var(unbiased=True) → Tensor

See torch.var()

vdot(other) → Tensor

See torch.vdot()

view(*shape) → Tensor

Returns a new tensor with the same data as the self tensor but of a different shape.

The returned tensor shares the same data and must have the same number of elements, but may have a different size. For a tensor to be viewed, the new view size must be compatible with its original size and stride, i.e., each new view dimension must either be a subspace of an original dimension, or only span across original dimensions \(d, d+1, \dots, d+k\) that satisfy the following contiguity-like condition that \(\forall i = d, \dots, d+k-1\),

\[\text{stride}[i] = \text{stride}[i+1] \times \text{size}[i+1]\]

Otherwise, it will not be possible to view self tensor as shape without copying it (e.g., via contiguous()). When it is unclear whether a view() can be performed, it is advisable to use reshape(), which returns a view if the shapes are compatible, and copies (equivalent to calling contiguous()) otherwise.

Parameters

shape (torch.Size or int...) – the desired size

Example:

>> x = torch.randn(4, 4)
>> x.size()
torch.Size([4, 4])
>> y = x.view(16)
>> y.size()
torch.Size([16])
>> z = x.view(-1, 8)  # the size -1 is inferred from other dimensions
>> z.size()
torch.Size([2, 8])

>> a = torch.randn(1, 2, 3, 4)
>> a.size()
torch.Size([1, 2, 3, 4])
>> b = a.transpose(1, 2)  # Swaps 2nd and 3rd dimension
>> b.size()
torch.Size([1, 3, 2, 4])
>> c = a.view(1, 3, 2, 4)  # Does not change tensor layout in memory
>> c.size()
torch.Size([1, 3, 2, 4])
>> torch.equal(b, c)
False

view(dtype) → Tensor

Returns a new tensor with the same data as the self tensor but of a different dtype. dtype must have the same number of bytes per element as self’s dtype.

Warning

This overload is not supported by TorchScript, and using it in a Torchscript program will cause undefined behavior.

Parameters

dtype (torch.dtype) – the desired dtype

Example:

>> x = torch.randn(4, 4)
>> x
tensor([[ 0.9482, -0.0310,  1.4999, -0.5316],
        [-0.1520,  0.7472,  0.5617, -0.8649],
        [-2.4724, -0.0334, -0.2976, -0.8499],
        [-0.2109,  1.9913, -0.9607, -0.6123]])
>> x.dtype
torch.float32

>> y = x.view(torch.int32)
>> y
tensor([[ 1064483442, -1124191867,  1069546515, -1089989247],
        [-1105482831,  1061112040,  1057999968, -1084397505],
        [-1071760287, -1123489973, -1097310419, -1084649136],
        [-1101533110,  1073668768, -1082790149, -1088634448]],
    dtype=torch.int32)
>> y[0, 0] = 1000000000
>> x
tensor([[ 0.0047, -0.0310,  1.4999, -0.5316],
        [-0.1520,  0.7472,  0.5617, -0.8649],
        [-2.4724, -0.0334, -0.2976, -0.8499],
        [-0.2109,  1.9913, -0.9607, -0.6123]])

>> x.view(torch.int16)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: Viewing a tensor as a new dtype with a different number of bytes per element is not supported.

view_as(other) → Tensor

View this tensor as the same size as other. self.view_as(other) is equivalent to self.view(other.size()).

Please see view() for more information about view.

Parameters

other (torch.Tensor) – The result tensor has the same size as other.

vsplit(split_size_or_sections) → List of Tensors

See torch.vsplit()

where(condition, y) → Tensor

self.where(condition, y) is equivalent to torch.where(condition, self, y). See torch.where()

xlogy(other) → Tensor

See torch.xlogy()

xlogy_(other) → Tensor

In-place version of xlogy()

zero_() → Tensor

Fills self tensor with zeros.

class borch.Module(posterior=None)

Bases: torch.nn.modules.module.Module

Acts as a torch.nn.Module but handles borch.RandomVariable s correctly.

It can be used in just the same way as in torch >>> import torch >>> import borch >>> class MLP(Module): … def __init__(self, in_size, out_size): … super().__init__() … self.fc1 = borch.nn.Linear(in_size, in_size*2) … self.relu = borch.nn.ReLU() … self.fc2 = borch.nn.Linear(in_size*2, out_size) … … def forward(self, x): … x = self.fc1(x) … x = self.relu(x) … x = self.fc2(x) … >>> mlp = MLP(2, 2) >>> out = mlp(torch.randn(3, 2))

It can be mixed with torch modules as one see fit. >>> class MLP2(Module): … def __init__(self, in_size, out_size): … super().__init__() … self.fc1 = torch.nn.Linear(in_size, in_size*2) … self.relu = torch.nn.ReLU() … self.fc2 = borch.nn.Linear(in_size*2, out_size) … … def forward(self, x): … x = self.fc1(x) … x = self.relu(x) … x = self.fc2(x) … >>> our = MLP2(2, 2)(torch.randn(3,2))

The more interesting case is when one start to involve borch.RandomVariable s >>> from borch import distributions as dist >>> from borch.posterior import Normal >>> class MyModule(Module): … def __init__(self, w_size): … super().__init__(posterior=Normal()) … self.weight = dist.Normal(torch.ones(w_size), torch.ones(w_size)) … … def forward(self, x): … return x.matmul(self.weight) … >>> my_module = MyModule(w_size=(4,))

Parameters

posterior – A borch.Posterior subclass that handles how the inference is preformed.

get(name)

Standard getattr with no custom overloading

property internal_modules

Get the internal modules borch uses, like prior, posterior, observed

observe(*args, **kwargs)

Set/revert any random variables on the current posterior to be observed /latent.

The behaviour of an observed variable means that any RandomVariable objects assigned will be observed at the stated value (if the name matches a previously observed variable).

Note

Calling observe' will overwrite all ``observe() calls made to ANY random variable attached to the module, even if it has a differnt name. One can still call observe on RandomVariable`` s in the forward after the observe call is made on the module.

Parameters

• args – If None, all observed behaviour will be forgotten.

• kwargs – Any named arguments will be set to observed given that the value is a tensor, or the observed behaviour will be forgotten if set to None.

Examples

>>> import torch
>>> from borch.distributions import Normal
>>> from borch.posterior import Automatic
>>>
>>> model = Module()
>>> rv = Normal(Tensor([1.]), Tensor([1.]))
>>> model.observe(rv_one=Tensor([100.]))
>>> model.rv_one = rv # rv_one has been observed
>>> model.rv_one
tensor([100.])
>>> model.observe(None)  # stop observing rv_one, the value is no
>>>                      # longer at 100.
>>> sample(model)
>>> torch.equal(model.rv_one, Tensor([100.]))
False

class borch.OptimizersCollection(optimizer, *args, **kwargs)

Bases: object

An organizer for a torch.optim.Optimize where one can add parameters dynamically.

It is designed to handel models where parameters is added in the forward and parameters are not guaranteed to be included in the forward pass.

Example

>>> import torch
>>> x = torch.tensor([10.], requires_grad=True)
>>> y = torch.tensor([10.], requires_grad=True)
>>> optimizer = OptimizersCollection(optimizer=torch.optim.Adam, lr=1)
>>> def objective(x, y):
...    return x*y

>>> for ii in range(2):
...     optimizer.zero_grad()
...     loss = objective(x, y)
...     loss.backward()
...     optimizer.step([x, y])

step(params, closure=None)

Preforms one optimizer step

Parameters

• params – The params to apply the step to

• closure – closure as for the torch.optim interface (Default value = None)

Returns

None

zero_grad()

Clears the gradients of all optimized torch.Tensor s.

class borch.RVPair(p_dist, q_dist)

Bases: borch.graph.Graph

Provide a prior and the corresponding approximating distribution.

This is useful when one wants a custom approximating distribution.

forward()

The forward

class borch.RandomVariable(validate_args=None, posterior=None)

Bases: borch.graph.Graph

Base class for a RandomVariable primitive used to model stochastic nodes. It merges a torch.tensor, torch.nn.Module and a torch.distribution.Distributions. It is not intended to be used directly but to be inherited from when creating a RandomVariable.

When used it will act as a torch.Tensor with a sample drawn from the distribution with most of the methods from a torch.nn.Module and a torch.distribution.Distributions

Examples

>>> import torch
>>> import borch

>>> class MyRV(RandomVariable):
...     distribution_cls = torch.distributions.Normal
...     def __init__(self, loc=1, scale=1):
...         super().__init__()
...         self.register_param_or_buffer("loc", loc)
...         self.register_param_or_buffer("scale", scale)
...         self()
...     def _distribution(self):
...         'Call that creates the torch distribution'
...         return self.distribution_cls(
...             borch.as_tensor(self.loc),
...             borch.as_tensor(self.scale)
...         )
>>> rv = MyRV(torch.nn.Parameter(torch.tensor(1.)), 2.)

It works as a normal tensor >>> torch.exp(rv) > 0 tensor(True)

One can also use the tensor attribute directly >>> type(rv.tensor) <class ‘torch.Tensor’> >>> torch.exp(rv) == torch.exp(rv.tensor) tensor(True)

The random variable have all the functionality from the torch.nn.Module >>> list(rv.parameters()) [Parameter containing: tensor(1., requires_grad=True)]

cdf(value=None)

Returns the cumulative density/mass function evaluated at value. :param value: :type value: Tensor

property distribution

Create the torch.distributions.Distribution equivalent

distribution_cls()

Creator for the torch.distributions.Distribution

entropy()

Returns entropy of distribution, batched over batch_shape.

Returns

Tensor of shape batch_shape.

forward()

Draw a sample from the distribution

property has_enumerate_support

support enumerate

property has_rsample

Check if rsample is implemented

icdf(value=None)

Returns the inverse cumulative density/mass function evaluated at value. :param value: :type value: Tensor

log_prob(value=None)

Returns the log of the probability density/mass function evaluated at value. :param value: :type value: Tensor

perplexity()

Returns perplexity of distribution, batched over batch_shape. :returns: Tensor of shape batch_shape.

rsample(sample_shape=torch.Size([]))

Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.

sample(sample_shape=torch.Size([]))

Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.

property support

Returns a :class:~torch.distributions.constraints.Constraint object representing this distribution’s support.

property validate_args

if the args should be validated when creating the distribution

class borch.Transform(transform, param, posterior=None)

Bases: borch.graph.Graph

Apply a transformation to a parameter

This can come in usefull in situations where one wants to apply some from of constraint for a parameter. For example, standard deviation of a normal distribution is not allowed to be negative. Therefore we use a Transform with transformation exp(param) and this transformed tensor will only be defined in the region (0, inf).

Examples

>>> import torch
>>> sin = Transform(torch.sin, torch.ones(2))
>>> torch.exp(sin)
tensor([2.3198, 2.3198])

The same thing can be acchived by nesting ``Transform``s >>> exp = Transform(torch.exp, sin) >>> exp*1 tensor([2.3198, 2.3198])

forward()

Apply the transformation

borch.as_tensor(val)

Convert val to a torch.Tensor if possible

Examples

>>> as_tensor(1.)
tensor(1.)
>>> as_tensor('hello')
'hello'

borch.named_random_variables(module, posterior=True, prior=False)

Get all random variables

borch.pq_dict(module) → dict

Create a dictionary where keys, values are prior distributions, approximating distributions, respectively.

Returns

List with dicts where each dict contains the information about the prior, posterior and value

Examples

>>> import torch
>>> net = borch.nn.Linear(1,2)

The pq_to_infer only returns information relating to RandomVariable s that has been accessed. So we just run the forward. >>> _= net(torch.randn(2, 1)) >>> p_q = pq_dict(net) >>> list(p_q[0].keys()) [‘prior’, ‘posterior’, ‘observed’, ‘value’]

borch.pq_to_infer(module)

Creats a dictionary of lists that can be used in borch.infer :returns: dictionary with keys corresponding to arguments in

infer.vi functions

Examples

>>> import torch
>>> net = borch.nn.Linear(1,2)

The pq_to_infer only returns information relating to RandomVariable s that has been accessed. So we just run the forward. >>> _= net(torch.randn(2, 1)) >>> p_q = pq_to_infer(net)

Then we can use the result to construct a loss >>> loss = borch.infer.vi_loss(**p_q) >>> loss.backward()

borch.random_variables(module, posterior=True, prior=False)

Generator to get all random variables from a network

borch.sample(module, posterior=True, prior=False, redraw=True, memo=None)

Sample all RandomVariable in a network

This is done by triggering a recalculation of each graph in the network

Examples

>>> import torch
>>> net = borch.nn.Linear(1,2)
>>> out= net(torch.randn(2, 1))
>>> sample(net)
>>> out == net(torch.randn(2, 1))
tensor([[False, False],
        [False, False]])

borch.set_posteriors(posterior_creator)

Function that takes any object and if it is a borch.nn.Module it sets the posterior to ``posterior_creator()’ on that object.

Parameters

posterior_creator (callable) – should return a borch.posterior.posterior

Examples

>>> from borch import posterior, nn, Module
>>> model = Module(posterior=posterior.Normal())
>>> model.lin = nn.Linear(10, 10)
>>> _ = model.apply(set_posteriors(posterior.Automatic))
>>> model.posterior
Automatic()

borch.validate_args(value)

Context manager that sets the validate_args for all random variable distributions.