When an IPI is generated by a CPU, the pattern looks roughly like:
<write shared data>
smp_wmb();
<write to GIC to signal SGI>
On the receiving CPU we rely on the fact that, once we've taken the
interrupt, then the freshly written shared data must be visible to us.
Put another way, the CPU isn't going to speculate taking an interrupt.
Unfortunately, this assumption turns out to be broken.
Consider that CPUx wants to send an IPI to CPUy, which will cause CPUy
to read some shared_data. Before CPUx has done anything, a random
peripheral raises an IRQ to the GIC and the IRQ line on CPUy is raised.
CPUy then takes the IRQ and starts executing the entry code, heading
towards gic_handle_irq. Furthermore, let's assume that a bunch of the
previous interrupts handled by CPUy were SGIs, so the branch predictor
kicks in and speculates that irqnr will be <16 and we're likely to
head into handle_IPI. The prefetcher then grabs a speculative copy of
shared_data which contains a stale value.
Meanwhile, CPUx gets round to updating shared_data and asking the GIC
to send an SGI to CPUy. Internally, the GIC decides that the SGI is
more important than the peripheral interrupt (which hasn't yet been
ACKed) but doesn't need to do anything to CPUy, because the IRQ line
is already raised.
CPUy then reads the ACK register on the GIC, sees the SGI value which
confirms the branch prediction and we end up with a stale shared_data
value.
This patch fixes the problem by adding an smp_rmb() to the IPI entry
code in gic_handle_irq. As it turns out, the combination of a control
dependency and an ISB instruction from the EOI in the GICv3 driver is
enough to provide the ordering we need, so we add a comment there
justifying the absence of an explicit smp_rmb().
Cc: stable@vger.kernel.org
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Adding EZchip NPS400 support.
Internal interrupts are handled by Multi Thread Manager (MTM)
Once interrupt is serviced MTM is acked for deactivating the interrupt.
External interrupts are handled by MTM as well as at Global Interrupt
Controller (GIC) e.g. serial and network devices.
Signed-off-by: Noam Camus <noamc@ezchip.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Vineet Gupta <vgupta@synopsys.com>
Acked-by: Jason Cooper <jason@lakedaemon.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Some kind of Freescale Layerscape SoC provides a MSI
implementation which uses two SCFG registers MSIIR and
MSIR to support 32 MSI interrupts for each PCIe controller.
The patch is to support it.
Signed-off-by: Minghuan Lian <Minghuan.Lian@nxp.com>
Tested-by: Alexander Stein <alexander.stein@systec-electronic.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The ACPI code requires to use global variables in order to collect
information from the tables.
To make clear those variables are ACPI specific, gather all of them in a
single structure.
Furthermore, even if some of the variables are not marked with
__initdata, they are all only used during the initialization. Therefore,
the new variable, which hold the structure, can be marked with
__initdata.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Currently, most of the pr_* messages in the GICv3 driver don't have a
prefix. Add one to make clear where the messages come from.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
For now, the firmware tables are parsed 2 times: once in the GIC
drivers, the other timer when initializing the vGIC. It means code
duplication and make more tedious to add the support for another
firmware table (like ACPI).
Introduce a new structure and set of helpers to get/set the virtual GIC
information. Also fill up the structure for GICv2.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
The ACPI code requires to use global variables in order to collect
information from the tables.
For now, a single global variable is used, but more will be added in a
subsequent patch. To make clear they are ACPI specific, gather all the
information in a single structure.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Acked-by: Christofer Dall <christoffer.dall@linaro.org>
Acked-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
We've unfortunately started seeing a situation where percpu interrupts
are partitioned in the system: one arbitrary set of CPUs has an
interrupt connected to a type of device, while another disjoint
set of CPUs has the same interrupt connected to another type of device.
This makes it impossible to have a device driver requesting this interrupt
using the current percpu-interrupt abstraction, as the same interrupt number
is now potentially claimed by at least two drivers, and we forbid interrupt
sharing on per-cpu interrupt.
A solution to this is to turn things upside down. Let's assume that our
system describes all the possible partitions for a given interrupt, and
give each of them a unique identifier. It is then possible to create
a namespace where the affinity identifier itself is a form of interrupt
number. At this point, it becomes easy to implement a set of partitions
as a cascaded irqchip, each affinity identifier being the HW irq.
This allows us to keep a number of nice properties:
- Each partition results in a separate percpu-interrupt (with a restrictied
affinity), which keeps drivers happy.
- Because the underlying interrupt is still per-cpu, the overhead of
the indirection can be kept pretty minimal.
- The core code can ignore most of that crap.
For that purpose, we implement a small library that deals with some of
the boilerplate code, relying on platform-specific drivers to provide
a description of the affinity sets and a set of callbacks.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: devicetree@vger.kernel.org
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Rob Herring <robh+dt@kernel.org>
Link: http://lkml.kernel.org/r/1460365075-7316-4-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Under the OX810SE, this exact same interface is used as "Reference Peripheral
Specification" Interrupt Controller, so add a new compatible string in order
to support the Oxford Semiconductor OX810SE SoC interrupt controller.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
When introducing the whole CPU feature detection framework,
we lost the capability to detect a mismatched GIC configuration
(using the GICv2 MMIO interface, but having the system register
interface enabled).
In order to solve this, use the new this_cpu_has_cap() helper.
Also move the check to the CPU interface path in order to catch
systems where the first CPU has been correctly configured,
but the secondaries are not.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Commit 2a07870511 ("irqchip/mips-gic: Use gic_vpes instead of
NR_CPUS") & commit 78930f09b9 ("irqchip/mips-gic: Clear percpu_masks
correctly when mapping") both introduce code which accesses gic_vpes
entries in the pcpu_masks array. However, this array has length NR_CPUS.
If NR_CPUS is less than gic_vpes (ie. the kernel supports use of less
CPUs than are present in the system) then we overrun the array, clobber
some other data & generally die pretty promptly.
Most notably this affects uniprocessor kernels running on any multicore
or multithreaded Malta with a GIC (ie. the vast majority of real Malta
boards).
Fix this by only accessing up to min(gic_vpes, NR_CPUS) entries in the
pcpu_masks array, preventing the array overrun.
Fixes: 2a07870511 ("irqchip/mips-gic: Use gic_vpes instead of NR_CPUS")
Fixes: 78930f09b9 ("irqchip/mips-gic: Clear percpu_masks correctly when mapping")
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Link: http://lkml.kernel.org/r/1461234714-9975-1-git-send-email-paul.burton@imgtec.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
In the add-on file for the GIC dealing with the RealView family
we currently only handle the PB11MPCore, let's extend this to
manage the RealView EB ARM11MPCore as well. The Revision B of the
ARM11MPCore core tile is a bit special and needs special handling
as it moves a system control register around at random.
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: devicetree@vger.kernel.org
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Always return IRQ_SET_MASK_OK_DONE instead of IRQ_SET_MASK_OK when the
affinity has been updated. When using stacked irqchips, returning
IRQ_SET_MASK_OK_DONE means skipping all descendant irqchips.
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
gicv3_init_bases() is the only caller for its_init(),
also it is a __init function, so mark its_init() as __init too,
then recursively mark the functions called as __init.
This will help to introduce ITS initialization using ACPI tables as
we will use acpi_table_parse_entries family functions there which
belong to __init section as well.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Following ACPI spec:
On systems supporting GICv3 and above, GICR Base Address in MADT GICC
structure holds the 64-bit physical address of the associated Redistributor.
If all of the GIC Redistributors are in the always-on power domain,
GICR structures should be used to describe the Redistributors instead,
and this field must be set to 0.
It means that we have two ways to initialize registirbutors map.
1. via GICD structure which can accommodate many redistributors as a region
2. via GICC which is able to describe single redistributor
This patch is going to add support for second option.
Considering redistributors, GICD and GICC subtables have be mutually
exclusive. While discovering and mapping redistributor, we need to know
its size in advance. For the GICC case, redistributor can be in
a power-domain that is off, thus we cannot relay on GICR TYPER register.
Therefore, we get GIC version from distributor register and map 2xSZ_64K
for GICv3 and 4xSZ_64K for GICv4.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
With the refator of gic_of_init(), GICv3/4 can be initialized
by gic_init_bases() with gic distributor base address and gic
redistributor region(s).
So get the redistributor region base addresses from MADT GIC
redistributor subtable, and the distributor base address from
GICD subtable to init GICv3 irqchip in ACPI way.
Note: GIC redistributor base address may also be provided in
GICC structures on systems supporting GICv3 and above if the GIC
Redistributors are not in the always-on power domain, this
patch didn't implement such feature yet.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Isolate hardware abstraction (FDT) code to gic_of_init().
Rest of the logic goes to gic_init_bases() and expects well
defined data to initialize GIC properly. The same solution
is used for GICv2 driver.
This is needed for ACPI initialization later.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Tomasz Nowicki <tomasz.nowicki@linaro.org>
Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Pull the second round of irqchip core changes for v4.6 from Jason Cooper:
- mvebu:
- Add odmi driver for Marvell 7K/8K SoCs
- Replace driver-specific set_affinity with generic version
- mips:
- Move ath79 MISC and CPU drivers from arch/ code to irqchip/
- tango:
- Add support for Sigma Designs SMP8[67]xx ctrl
Pull irqchip core changes for v4.6 from Jason Cooper:
- mvebu (armada-370-xp)
- MSI support
- Deconflict with mvebu's arm64 code
- ts4800
- Restrict when ts4800 driver can be built
- Make ts4800_ic_ops static const
- bcm2836: Drop superfluous memory barrier
Function its_alloc_tables() maintains two local variables, "order" and
and "alloc_size", to hold memory size that has been allocated to
ITS_BASEn. We don't always refresh the variable alloc_size whenever
value of the variable order changes, causing the following two
problems.
- Cache flush operation with size more than required.
- Information reported by pr_info is not correct.
Use a helper macro that converts page order to size in bytes instead of
variable "alloc_size" to fix both the problems.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The driver stays the same but the initialization changes a bit.
For OF boards we now get the memory map from the OF node and use
a linear mapping instead of the legacy mapping. For legacy boards
we still use a legacy mapping and just pass down all the parameters
from the board init code.
Signed-off-by: Alban Bedel <albeu@free.fr>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1453553867-27003-1-git-send-email-albeu@free.fr
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
The irq-armada-370-xp driver can only be built for ARM 32 bits. The mvebu
family had grown with a new ARM64 SoC which will also select the
ARCH_MEVBU configuration. Since "ARM: mvebu: use the ARMADA_370_XP_IRQ
option", the ARM32 mvebu SoC directly select this new option. Selecting
it by default when ARCH_MEVBU is selected is no more needed.
This patch removes this dependency, thanks to this, a kernel for ARM64
mvebu SoC can be built without error due this driver.
Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
Link: https://lkml.kernel.org/r/1454951660-13289-3-git-send-email-gregory.clement@free-electrons.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
