When a scheduler message starts porcessing,we start
the scheduler watchdog timer, and in few cases when the
system get under heavy load processing tasklets
and bottom halves, and the scheduler message is preempted,
its not getting a chance to run and the scheduler watchdog
timer is triggered and it causes a panic.
Since the scheduler watchdog timmer is less than
the kernel RCU timer, were are having scenarios, where
because of the scheduler watchdog panic, the RCU stall is masked.
So in case of scheduler watchdog we wanted to just print
the information about the message being processed by the scheduler
and not panic.
Change-Id: I509abcd8cd8d2e863dc3f2370ceaa70e0d4f244f
CRs-Fixed: 3004176
The scheduler watchdog timeout is currently fixed at 10secs.
There are few customer use cases, where the CPU is busy for
sometime and the scheduler message processing is taking longer than
the coinfigured watchdog timeout value.
Add a new API to be able to configure this value different than the
default as per the requirement.
CRs-Fixed: 2834194
Change-Id: I976f1f0ad17f09cf3960f99ad80226d775b8b2f2
During the module initialization the scheduler thread and the
init thread run in parallel if there is a failure in the error
path, the scheduler thread is not aware and still sending
the messages to firmware resulting in system instability.
So, Disable the scheduler as soon as there is an error in the init path
and the same scheduler thread in the normal existing error path.
If the scheduler_disable is called multiple times the infinite wait
for result will get the init thread struck.
Change-Id: I39782bc88d49dac2e2138c235c4666ea4f4f2faf
CRs-Fixed: 2754023
This fixes a CFI failure in callback assignment.
Currently, the callbacks in scheduler_msg are void function
pointers. Update them to have a defined type as
scheduler_msg_process_fn_t to catch type mismatch during
compilation.
Other changes to conform to this new type include:
1. Cast callback to qdf_mc_timer_callback_t when
referencing, and back when assigning
2. Cast wlan_serialization_generic_timer_cb to
take fit into scheduler_msg
3. Cast target_if_vdev_mgr_rsp_timer_cb to fit
into scheduler_msg
Change-Id: I052bc54826d377ae92f5bcc80ca08afb6f5e01e3
CRs-fixed: 2719975
For some targets, whenever the recovery is triggered in the scheduler
thread context, in the same context the firmware dump collection
could also happen and can more time than the watchdog timeout for
the same.
So check and ignore watchdog timeout when the recovery is already
inprogress.
Change-Id: I0e74e73139a83c6361407fe2eef8d6d0cf2a5d8c
CRs-Fixed: 2718984
The scheduler maintains a history of the recent messages
in the scheduler that were queued, started or completed execution.
This information can be reviewed while debugging issues
if the scheduler is taking longer to process any message.
The new API will print the scheduler history on request.
Change-Id: I2ed7989296f6891cfd9779e7b96383253d680f51
CRs-Fixed: 2599281
The scheduler watchdog timer handler currently either prints error or
prints error and panics based on CONFIG_SLUB_DEBUG_ON.
Considering the panic through QDF_DEBUG_PANIC is configurable and is
disabled on relevant builds, we can just have one version of the
function and depend on QDF macro for panic and dependency on
CONFIG_SLUB_DEBUG_ON is redundant
Change the default log levels from none to fatal for scheduler.
Change-Id: If588455ea2779311eb86bb39b3ab0f7fc67b6ccc
CRs-Fixed: 2593790
Structure memory is being optimized by aligning field with
bus address length using pokeahole tool.
This patch optimizes structure linked in obj_mgr and scheduler
modules.
Structure Bytes saved (64bit) Bytes saved (32bit)
scheduler_ctx 8 bytes 4 bytes
wlan_channel 8 bytes 8 bytes
Change-Id: I298946b55328d8d9b564c140e0a59e0b8702fab0
CRs-Fixed: 2587842
Add support to export scheduler_post_message_debug API
to be used in wmi_unified.c
Change-Id: If18741f1973724453f0b2de12e133230924a2e52
CRs-Fixed: 2555961
Remove CONFIG_MCL for Serialization msg queue depth and reduction limit to
make code generic. In case SCHEDULER_CORE_MAX_MESSAGES not defined, then
define SCHEDULER_CORE_MAX_MESSAGES to 4000 for WIN . If
WLAN_SCHED_REDUCTION_LIMIT not defined for MCL then define
WLAN_SCHED_REDUCTION_LIMIT to 32 for WIN .
Change-Id: If64f117f6f0af56f6a9e1041e583b022c6c419fb
CRs-Fixed: 2457723
Due to a limitation in the memory leak detection logic when qdf_flex_mem
was written, qdf_flex_mem_release() was necessary to free any
dynamically allocated memory in the qdf_flex_mem_pool during certain
points in the WLAN driver lifecycle. This was done to avoid flagging any
dynamically allocated qdf_flex_mem_segments as false positive leaks
when the memory domains were changed and leak checking was performed.
With the introduction of qdf_talloc(), this workaround is no longer
required. Replace the calls to qdf_mem_malloc/free with qdf_talloc/free,
and remove qdf_flex_mem_release() altogether.
Change-Id: Ia5fd21386b94fc117af5f27853db5d8341601738
CRs-Fixed: 2404955
There are some changes to timer APIs in latest kernel,
update driver APIs accordingly to invoke correct kernel
APIs for timer functionalities.
Change-Id: Ie017c8b1ef8237ca34f696c23509519a1187167c
CRs-fixed: 2383574
Since the usage of the scheduler for WIN,
is now not limited only to scan component alone and
would be used by the new converged MLME also for few specific cases,
we need to increase the size of max scheduler buffers to 4K.
Change-Id: I11818c311b39c42ffda5a79eb7b7137dcb491649
CRs-Fixed: 2387644
Add mlme message queue handler as the call back for non scan commands
getting activated in the scheduler context.
Add cmd already exists enum to be returned if the command already exists
in the serialization queue.
Change-Id: I24fe453ffddbd4341459874458f11688adebc740
CRs-Fixed: 2377219
The scheduler history feature contains logic to capture the queue depth
at the time of message insertion, but an oversight means this
information is not actually recorded in the scheduler history. Record
the captured queue depth in the scheduler history when a message is
dequeued for processing.
Change-Id: Ia2dd6f641055237b8f9c546077f7b5992c697330
CRs-Fixed: 2332752
In order to triage latency issues in the scheduler thread, track which
queue a message was queue in, when each message was queued, how long it
spent in queue, and the depth of the queue at the point of insertion.
Change-Id: I83edd04e668d5a2674ca71495fc904092b3ab800
CRs-Fixed: 2322561
The scheduler can run out of buffers and we panic
whenever this happens. However in the older code
we wait until the failure count of getting scheduler
buffers reaches a defined limit, thus making buffers
available in due time and the panic was seen in rare cases,
where failure count was exceeding the defined limit.
With the qdf_flex_mem supprt added for scheduler, the
wait to panic, when the failure count was removed and
we panic when the first failure to get scheduler buffer
is reached. This change adds to wait until the failure
count reaches the limit and then panic.
Change-Id: Ie8774830dc0cefdfccb1473216a9300609ab65bb
CRs-Fixed: 2322049
In the scheduler_post_message, src_id is now added to
know the source module of the msg. the present scheduler
doesn't know about the same which is scheduler_post_msg
Replace the scheduler_post_msg with scheduler_post_message
Change-Id: I795f5d714eebcdbb275daae782ffa6f4d8e7e950
CRs-Fixed: 2306019
Trace every msg, and print its src, and dest in
scheduler_post_msg_by_priority for ML logging
Print to F3, and trace it too
Change-Id: I64a2142bf3277e85e970132933f2157b95dad68d
CRs-Fixed: 2306043
Currently the scheduler cant differentiate between
the destination, and que to which it has to post the
message.
Add que_id, to differentiate between the destination
and que.
Change-Id: I94f81cb4b976dba4571f9cd0d6e059dc72024fcd
CRs-Fixed: 2308108
To help triage issues where the scheduler thread is processing slowly,
add a scheduler processing history for offline analysis. This can also
be useful in determining if a processing delay in some operation is
attributed to the scheduler thread or not.
Change-Id: Icfea17efc34b2a23d4f8b2baebe08db4e2f6ce8b
CRs-Fixed: 2307252
Currently the scheduler is unaware of the source module
which posts the message to another layer.
To trace the src, dest, and the message under ML
logging, modify current infra, to include the src,
dest in the qip argument of the API
scheduler_post_msg_by_priority
Change-Id: Ia234f60cc33fcc37ab741c462052a6e50993e3bd
CRs-Fixed: 2300537
Drop beacon/probe frames before posting on Scan queue if the queue
already has too many beacon/probes to process.
Also add scheduler API to get the queue size given the module ID.
Change-Id: I9153c7e77e74377863774b68e8163839e992358d
CRs-Fixed: 2298584
Add per module logging macros without function/line info
to avoid adding function/line info where it is not required.
Change-Id: I34ba6f6485f9b506264daf4e116052d2858bed40
CRs-Fixed: 2278876
As part of the effort to reduce overly verbose and excessive logging
driver wide, reduce or remove logs in scheduler as much as possible.
Change-Id: I1e45c439315e81d9ff05e1688fe690d0ca30e097
CRs-Fixed: 2280337
Map all module level log APIs to per-level log APIs
to compile out specific log level if required.
Change-Id: I4072b6740cb43200fd95c40943b66e1d5f8f1847
CRs-Fixed: 2266719
QDF_DEBUG_PANIC has recently gained a reason parameter. Move instance of
logging followed immediately by QDF_DEBUG_PANIC to single calls.
Change-Id: Ifc00be410621005e8494074ed00b1b7e44cc6a03
CRs-Fixed: 2271769
Currently, the scheduler thread keeps a large, pre-allocated array of
messages for use in message posting. The vast majority of the time,
however, the scheduler thread has zero or one messages pending in the
queue. This leads to a huge memory overhead for nominal driver
operation.
Replace the current pre-allocated scheduler message pool with a
hybrid static/dynamic approach.
Change-Id: Ie942bacfef43edf142a9f35ad0309069096cda90
CRs-Fixed: 2204172
Configure the qdf_flex_mem segments for scheduler pool
to 32, which creates 1024 scheduler message buffers
during init.
Change-Id: Iefa9427b55b427f79aa21a90cbf6e8a05be35e59
CRs-Fixed:
Address the following issue in scheduler_api.h:
CHECK: 'mesage' may be misspelled - perhaps 'message'?
Change-Id: Iaf5138f335be1dad2db2e3c52ffd47e28da21251
CRs-Fixed: 2241581
A global timer is created and freed by each module, if passed as
msg to mc queues, shouldn't be freed in scheduler_cleanup_queues,
or double free issue happens
For example, a timer in tLimTimers is freed in mac_stop->pe_stop
->lim_cleanup_mlm
Change-Id: I919b13f2570e3496a1de249fd90a88f3bfeaa131
CRs-Fixed: 2220514
Add scheduler message handler function for QDF_MODULE_ID_SCAN queue.
Also post scan beacon/probe frames and scan events to SCAN queue instead
of TARGET_IF queue.
Change-Id: I94e3ea2f0eb5c9592f5cfcd91738778fa65ba815
CRs-Fixed: 2214554
After profiling many iterations of several driver operations, the
observed absolute maximum number of queue scheduler messages was 452
during soft AP startup. Typical maximums for this test case are less
than 100, occasionally getting as high as 250. The median for all of
the driver operations in general were in the range of 1-2. Given this
information, a round number around twice the worst case rate was
picked.
Reduce SCHEDULER_CORE_MAX_MESSAGES to 1000, down from 8000 for MCL.
This saves a significant amount of unused statically allocated memory.
Change-Id: I6ef65db84ca524cea68afbc3bb49347c6d996992
CRs-Fixed: 2202692
Part of the scheduler component has been migrated to the new sched_*
logging wrappers, but part of it still remains to be ported. Finish
migrating all of the scheduler logging APIs.
Change-Id: I5065d849a1523d6c27d6c6a27b1ad4605036c3e6
CRs-Fixed: 2201038
Reduced the number of scheduler message buffers for WIN to
2k from 4k since WIN is only using 2 scheduler queues.
Changed the order of two members of scheduler messgae structure
to avoid additional padding on 64bit platform.
Change-Id: Ic74d60878867c45413bc4fbd5c56feb4dd421699
CRs-Fixed: 2157621
A commonly used and robust life-cycle pattern consists of the following
4 steps:
1) create (allocate resources)
2) start (mark resources as safe for use)
3) stop (mark resources as unsafe for use)
4) destroy (deallocate resources)
This pattern effectively prevents access to uninitialized and freed
resources. While the dispatcher psoc life-cycle follows this pattern,
the global life-cycle is condensed into steps 1 and 4. Unsurprisingly,
this led to uninitialized and freed resource access problems, especially
in regards to the scheduler thread.
Split the scheduler init and deinit life-cycle functions into init,
enable, disable, and deinit. Create new global enable and disable
dispatcher functions, and call the new scheduler APIs as appropriate.
This brings the global dispatcher life-cycle in line with the existing
psoc life-cycle, and prevents many scheduler related resource access
issues.
Change-Id: I58b65be0611f48e48354f28c221185f6f490f30c
CRs-Fixed: 2153283
There are many minor discrepancies in the scheduler module's init and
deinit paths. Refactor the scheduler init/deinit code paths to address
any discrepancies between the two. Also ensure error path cleanup cleans
up all the resources that were previously created.
Change-Id: Ib392f41a887031cd710aebaca32eb653952550a4
CRs-Fixed: 2153210
A set of logging helpers for the scheduler component were previously
added to reduce boilerplate logging code. Update scheduler_api.c to use
the new logging helpers.
Change-Id: I45715e40dbdd5a94d064ca4c64290a443cacea34
CRs-Fixed: 2152575
The basic qdf bit ops are not atomic. Race conditions
can lead to messages stuck in queues without mc thread
being woken up.
Change-Id: I42093b863c7ed93c5c2baf0bd14106eb4fe082fc
CRs-Fixed: 2118014
In the event of a scheduler thread watchdog bite, dump the scheduler
thread stack trace to aid in debugging.
Change-Id: I222556c7cb01985abe20b04a9383fc6c6d0724fc
CRs-Fixed: 2091875
The SYS_MSG_ID_MC_START and SYS_MSG_ID_MC_STOP message IDs are no
longer being used. so remove them.
Change-Id: I8d35a537ac2c2f7436b23033561e5d0ef09895df
CRs-Fixed: 2107075
