The memset and scatter gathered memset are going to use some common logic
to create their descriptors.
Move that logic into a function of its own so that we can share it with the
future memset_sg callback.
Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: Nicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
In kernel-doc annotations parameters need to start with a @ for them to be
properly recognized. Add those where missing for virt-dma.
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
With the old binding and driver architecture we had many issues:
No way to assign eDMA channels to event queues, thus not able to tune the
system by moving specific DMA channels to low/high priority servicing. We
moved the cyclic channels to high priority within the code, but that was
just a workaround to this issue.
Memcopy was fundamentally broken: even if the driver scanned the DT/devices
in the booted system for direct DMA users (which is not effective when the
events are going through a crossbar) and created a map of 'used' channels,
this information was not really usable. Since via dmaengien API the eDMA
driver will be called with _some_ channel number, we would try to request
this channel when any channel is requested for memcpy. By luck we got
channel which is not used by any device most of the time so things worked,
but if a device would have been using the given channel, but not requested
it, the memcpy channel would have been waiting for HW event.
The old code had the am33xx/am43xx DMA event router handling embedded. This
should have been done in a separate driver since it is not part of the
actual eDMA IP.
There were no way to 'lock' PaRAM slots to be used by the DSP for example
when booting with DT.
In DT boot the edma node used more than one hwmod which is not a good
practice and the kernel prints warning because of this.
With the new bindings and the changes in the driver we can:
- No regression with Legacy binding and non DT boot
- DMA channels can be assigned to any TC (to set priority)
- PaRAM slots can be reserved for other cores to use
- Dynamic power management for CC and TCs, if only TC0 is used all other TC
can be powered down for example
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Since the crossbar is needed for eDMA when it is used on OMAP like
platforms (am335x/am437x and later DRA7xx), select the crossbar to be built
if ARCH_OMAP is set.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The DMA event crossbar on AM33xx/AM43xx is different from the one found in
DRA7x family.
Instead of a single event crossbar it has 64 identical mux attached to each
eDMA event line. When the 0 event mux is selected, the default mapped event
is going to be routed to the corresponding eDMA event line. If different
mux is selected, then the selected event is going to be routed to the given
eDMA event.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Instead of nesting functions just merge them since the resulting function
is still small and readable.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The channel/slot reservation is not supported when booted with DT so there
is not need to allocate memory.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Move all code under one function to do the dma device and eDMA channel
related setup so they are not scattered around the driver.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
edma_assign_channel_eventq() is a wrapper around edma_map_dmach_to_queue()
We can merge the content of the later so we will have only one function
to be used for mapping channels to given eventq
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
These inline functions are designed to modify parts of the PaRAM in eDMA.
Change the names accordingly.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Instead of passing a pointer to struct edma_cc and the channel number,
pass only the pointer to the edma_chan structure for the given channel.
This struct contains all the information needed by the functions and the
use of this makes it obvious that most of the sanity checks can be removed
from the driver.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
If the transfer is shorted then 64K we can complete it with one ACNT burst
by configuring ACNT to the length of the copy, this require one paRAM slot.
Otherwise we use two paRAM slots for the copy:
slot1: will copy (length / 32767) number of 32767 byte long blocks
slot2: will be configured to copy the remaining data.
According to tests this patch increases the throughput of memcpy from
~3MB/s to 15MB/s
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Despite the claim by the original commit adding the memcpy
support, eDMA does not have constraint on the alignment of src, dst
or length in increment mode.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
HSU (High Speed UART) DMA engine, like the name suggests, is
an integrated DMA engine for UART and UART alone. Therefore,
making the UART drivers responsible of selecting it and
removing the user selectable option for it. The UARTs with
this DMA engine can always select HSU_DMA when
SERIAL_8250_DMA option is enabled.
Suggested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Heikki Krogerus <heikki.krogerus@linux.intel.com>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The DMA engine supports memory copy, RAID5 XOR, RAID6 PQ, and other
computations. But the bandwidth of the entire DMA engine is shared
among all channels. This patch re-configures operations availability
such that one can achieve maximum performance for XOR and PQ
computation by removing the memory offload operations.
Signed-off-by: Rameshwar Prasad Sahu <rsahu@apm.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
If the eDMA3 has support for channel paRAM slot mapping we can utilize it
to allocate slots on demand and save precious slots for real transfers.
On am335x the eDMA has 64 channels which means we can unlock 64 paRAM
slots out from the available 256.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The names chosen for the bitfields were quite confusing and given no real
information on what they are used for...
edma_inuse -> slot_inuse: tracks the slot usage/availability
edma_unused -> channel_unused: tracks the channel usage/availability
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Instead of directly reading it from CCCFG register take the information out
once when we set up the configuration from the HW.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
No need to run through the bits in QEMR and CCERR events since they will
not trigger any action, so just clearing the errors there is fine.
In case of the missed event the loop can be optimized so we spend less time
to handle the event.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
In the ccerr interrupt handler the code checks for pending errors in the
error status registers in two different places.
Move the check out to a helper function.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
With the merger of the arch/arm/common/edma.c code into the dmaengine
driver, there is no longer need to have per channel callback/data storage
for interrupt events.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Warning message in case of linking between paRAM slots in different eDMA
controllers.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
We have access to dev, so it is better to use the dev_dbg for debug prints.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Be consistent and do not mix the use of dev, &pdev->dev, etc in the
functions.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
When allocating a memory for number of items it is better (looks better)
to use devm_kcalloc.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Instead of using defines to specify the size of different arrays and
bitmaps, allocate the memory for them based on the information we get from
the HW itself.
Since these defines are set based on the worst case, there are devices
where they are not valid.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Move the code out from arch/arm/common and merge it inside of the dmaengine
driver.
This change is done with as minimal (if eny) functional change to the code
as possible to avoid introducing regression.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Since the driver stack no longer depends on lookup with id number in a
global array of pointers, the limitation for the number of eDMAs are no
longer needed. We can handle as many eDMAs in legacy and DT boot as we have
memory for them to allocate the needed structures.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Instead of relying on indexes pointing to edma private date in the global
pointer array, pass the private data pointer via the public API.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
If the of_dma_controller is registered in the non dmaengine driver we could
have race condition:
the of_dma_controller has been registered, but the dmaengine driver is not
yet probed. Drivers requesting DMA channels during this window will fail
since we do not yet have dmaengine drivers registered.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The code path in edma_execute() and edma_callback() can be simplified
and make it more optimal.
There is not need to call in to edma_execute() when the transfer
has been finished for example.
Also the handling of missed/first or next batch of paRAMs can
be done in a more optimal way.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
There is no need to print that the driver has been initialized
or removed, so remove such messages.
Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Since commit d078cd1b41 ("dmaengine: imx-sdma: Add imx6sx platform
support") we get this message on every boot on mx6q:
imx-sdma 20ec000.sdma: no event needs to be remapped
, which is not very helpful.
Move the message to debug level instead.
Cc: Zidan Wang <zidan.wang@freescale.com>
Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
This is needed due to the duplicated iommu stuff to help with the merge
and to prevent future issues.
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The L3 throughput can be higher than expected when packed access
is not enabled. The ratio depends on the number of bytes in a
transaction and the EMIF interface width.
The throughput was measured for the following settings/cases:
* Case 1: Burst size of 64 bytes, packed access disabled
* Case 2: Burst size of 64 bytes, packed access enabled
* Case 3: Burst disabled, packed access disabled
Throughput measurements were done during McASP-based audio
playback on the Jacinto6 EVM using the omapconf tool [1]:
$ omapconf trace bw -m sdma_rd
---------------------------------------------------------
Throughput (MB/s)
Audio parameters Case 1 Case 2 Case 3
---------------------------------------------------------
44.1kHz, 16-bits, stereo 1.41 0.18 1.41
44.1kHz, 32-bits, stereo 1.41 0.35 1.41
44.1kHz, 16-bits, 4-chan 2.82 0.35 2.82
44.1kHz, 16-bits, 6-chan 4.23 0.53 4.23
44.1kHz, 16-bits, 8-chan 5.64 0.71 5.64
---------------------------------------------------------
From above measurements, case 2 is the only one that delivers
the expected throughput for the given audio parameters. For
that reason, the packed accesses are now enabled.
It's worth to mention that packed accesses cannot be enabled
for all addressing modes. In cyclic transfers, it can be
enabled in the source for MEM_TO_DEV and in dest for DEV_TO_MEM,
as they use post-increment mode which supports packed accesses.
Peter Ujfalusi:
From the TRM regarding to this:
"NOTE: Except in the constant addressing mode, the source or
destination must be specified as packed for burst transactions
to occur."
So w/o the packed setting the burst on the MEM side was not
enabled, this explains the numbers.
[1] https://github.com/omapconf/omapconf
Signed-off-by: Misael Lopez Cruz <misael.lopez@ti.com>
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The MIC X100 DMA engine has a special status descriptor which writes
an 8 byte value to a destination location. This is used to signal
completion of all DMA descriptors prior to the status descriptor.
This patch add a new DMA engine API which enables updating a
destination address with an 8 byte immediate data value.
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Lawrynowicz, Jacek <jacek.lawrynowicz@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Siva Yerramreddy <yshivakrishna@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Pull dmaengine fixes from Vinod Koul:
"This contains fixes spread throughout the drivers, and also fixes one
more instance of privatecnt in dmaengine.
Driver fixes summary:
- bunch of pxa_dma fixes for reuse of descriptor issue, residue and
no-requestor
- odd fixes in xgene, idma, sun4i and zxdma
- at_xdmac fixes for cleaning descriptor and block addr mode"
* tag 'dmaengine-fix-4.3-rc4' of git://git.infradead.org/users/vkoul/slave-dma:
dmaengine: pxa_dma: fix residue corner case
dmaengine: pxa_dma: fix the no-requestor case
dmaengine: zxdma: Fix off-by-one for testing valid pchan request
dmaengine: at_xdmac: clean used descriptor
dmaengine: at_xdmac: change block increment addressing mode
dmaengine: dw: properly read DWC_PARAMS register
dmaengine: xgene-dma: Fix overwritting DMA tx ring
dmaengine: fix balance of privatecnt
dmaengine: sun4i: fix unsafe list iteration
dmaengine: idma64: improve residue estimation
dmaengine: xgene-dma: fix handling xgene_dma_get_ring_size result
dmaengine: pxa_dma: fix initial list move
A very tiny temporal window exists in the residue calculation where :
- upon entering residue calculation, the transfer is ongoing
- when reading the current transfer pointer, it just changed to
the "finisher/linker" descriptor
In this case, the residue returned is the whole transfer length instead
of 0. Fix it.
This appears almost in one extreme case, where the driver is used
by older clients which inquire for residue in interrupt context, such
as the smsc91x ethernet driver, in a tight loop :
interrupt_handler()
dmaengine_submit()
do {
dmaengine_tx_status()
} while (residue > 0 || status != DMA_ERROR)
Signed-off-by: Robert Jarzmik <robert.jarzmik@free.fr>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
A very small number of devices don't use the flow control offered by
requestor lines. In these specific cases, the pxa dma driver should be
aware of that and not try to use a requestor line.
Signed-off-by: Robert Jarzmik <robert.jarzmik@free.fr>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
This platform driver has a OF device ID table but the OF module
alias information is not created so module autoloading won't work.
Signed-off-by: Luis de Bethencourt <luisbg@osg.samsung.com>
Acked-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
This platform driver has a OF device ID table but the OF module
alias information is not created so module autoloading won't work.
Signed-off-by: Luis de Bethencourt <luisbg@osg.samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
