When a scheduler message starts porcessing,we start
the scheduler watchdog timer, and in few cases when the
system get under heavy load processing tasklets
and bottom halves, and the scheduler message is preempted,
its not getting a chance to run and the scheduler watchdog
timer is triggered and it causes a panic.
Since the scheduler watchdog timmer is less than
the kernel RCU timer, were are having scenarios, where
because of the scheduler watchdog panic, the RCU stall is masked.
So in case of scheduler watchdog we wanted to just print
the information about the message being processed by the scheduler
and not panic.
Change-Id: I509abcd8cd8d2e863dc3f2370ceaa70e0d4f244f
CRs-Fixed: 3004176
The scheduler watchdog timeout is currently fixed at 10secs.
There are few customer use cases, where the CPU is busy for
sometime and the scheduler message processing is taking longer than
the coinfigured watchdog timeout value.
Add a new API to be able to configure this value different than the
default as per the requirement.
CRs-Fixed: 2834194
Change-Id: I976f1f0ad17f09cf3960f99ad80226d775b8b2f2
During the module initialization the scheduler thread and the
init thread run in parallel if there is a failure in the error
path, the scheduler thread is not aware and still sending
the messages to firmware resulting in system instability.
So, Disable the scheduler as soon as there is an error in the init path
and the same scheduler thread in the normal existing error path.
If the scheduler_disable is called multiple times the infinite wait
for result will get the init thread struck.
Change-Id: I39782bc88d49dac2e2138c235c4666ea4f4f2faf
CRs-Fixed: 2754023
This fixes a CFI failure in callback assignment.
Currently, the callbacks in scheduler_msg are void function
pointers. Update them to have a defined type as
scheduler_msg_process_fn_t to catch type mismatch during
compilation.
Other changes to conform to this new type include:
1. Cast callback to qdf_mc_timer_callback_t when
referencing, and back when assigning
2. Cast wlan_serialization_generic_timer_cb to
take fit into scheduler_msg
3. Cast target_if_vdev_mgr_rsp_timer_cb to fit
into scheduler_msg
Change-Id: I052bc54826d377ae92f5bcc80ca08afb6f5e01e3
CRs-fixed: 2719975
For some targets, whenever the recovery is triggered in the scheduler
thread context, in the same context the firmware dump collection
could also happen and can more time than the watchdog timeout for
the same.
So check and ignore watchdog timeout when the recovery is already
inprogress.
Change-Id: I0e74e73139a83c6361407fe2eef8d6d0cf2a5d8c
CRs-Fixed: 2718984
The scheduler maintains a history of the recent messages
in the scheduler that were queued, started or completed execution.
This information can be reviewed while debugging issues
if the scheduler is taking longer to process any message.
The new API will print the scheduler history on request.
Change-Id: I2ed7989296f6891cfd9779e7b96383253d680f51
CRs-Fixed: 2599281
The scheduler watchdog timer handler currently either prints error or
prints error and panics based on CONFIG_SLUB_DEBUG_ON.
Considering the panic through QDF_DEBUG_PANIC is configurable and is
disabled on relevant builds, we can just have one version of the
function and depend on QDF macro for panic and dependency on
CONFIG_SLUB_DEBUG_ON is redundant
Change the default log levels from none to fatal for scheduler.
Change-Id: If588455ea2779311eb86bb39b3ab0f7fc67b6ccc
CRs-Fixed: 2593790
Add support to export scheduler_post_message_debug API
to be used in wmi_unified.c
Change-Id: If18741f1973724453f0b2de12e133230924a2e52
CRs-Fixed: 2555961
Remove CONFIG_MCL for Serialization msg queue depth and reduction limit to
make code generic. In case SCHEDULER_CORE_MAX_MESSAGES not defined, then
define SCHEDULER_CORE_MAX_MESSAGES to 4000 for WIN . If
WLAN_SCHED_REDUCTION_LIMIT not defined for MCL then define
WLAN_SCHED_REDUCTION_LIMIT to 32 for WIN .
Change-Id: If64f117f6f0af56f6a9e1041e583b022c6c419fb
CRs-Fixed: 2457723
Due to a limitation in the memory leak detection logic when qdf_flex_mem
was written, qdf_flex_mem_release() was necessary to free any
dynamically allocated memory in the qdf_flex_mem_pool during certain
points in the WLAN driver lifecycle. This was done to avoid flagging any
dynamically allocated qdf_flex_mem_segments as false positive leaks
when the memory domains were changed and leak checking was performed.
With the introduction of qdf_talloc(), this workaround is no longer
required. Replace the calls to qdf_mem_malloc/free with qdf_talloc/free,
and remove qdf_flex_mem_release() altogether.
Change-Id: Ia5fd21386b94fc117af5f27853db5d8341601738
CRs-Fixed: 2404955
There are some changes to timer APIs in latest kernel,
update driver APIs accordingly to invoke correct kernel
APIs for timer functionalities.
Change-Id: Ie017c8b1ef8237ca34f696c23509519a1187167c
CRs-fixed: 2383574
Add mlme message queue handler as the call back for non scan commands
getting activated in the scheduler context.
Add cmd already exists enum to be returned if the command already exists
in the serialization queue.
Change-Id: I24fe453ffddbd4341459874458f11688adebc740
CRs-Fixed: 2377219
The scheduler history feature contains logic to capture the queue depth
at the time of message insertion, but an oversight means this
information is not actually recorded in the scheduler history. Record
the captured queue depth in the scheduler history when a message is
dequeued for processing.
Change-Id: Ia2dd6f641055237b8f9c546077f7b5992c697330
CRs-Fixed: 2332752
In order to triage latency issues in the scheduler thread, track which
queue a message was queue in, when each message was queued, how long it
spent in queue, and the depth of the queue at the point of insertion.
Change-Id: I83edd04e668d5a2674ca71495fc904092b3ab800
CRs-Fixed: 2322561
The scheduler can run out of buffers and we panic
whenever this happens. However in the older code
we wait until the failure count of getting scheduler
buffers reaches a defined limit, thus making buffers
available in due time and the panic was seen in rare cases,
where failure count was exceeding the defined limit.
With the qdf_flex_mem supprt added for scheduler, the
wait to panic, when the failure count was removed and
we panic when the first failure to get scheduler buffer
is reached. This change adds to wait until the failure
count reaches the limit and then panic.
Change-Id: Ie8774830dc0cefdfccb1473216a9300609ab65bb
CRs-Fixed: 2322049
In the scheduler_post_message, src_id is now added to
know the source module of the msg. the present scheduler
doesn't know about the same which is scheduler_post_msg
Replace the scheduler_post_msg with scheduler_post_message
Change-Id: I795f5d714eebcdbb275daae782ffa6f4d8e7e950
CRs-Fixed: 2306019
Trace every msg, and print its src, and dest in
scheduler_post_msg_by_priority for ML logging
Print to F3, and trace it too
Change-Id: I64a2142bf3277e85e970132933f2157b95dad68d
CRs-Fixed: 2306043
Currently the scheduler cant differentiate between
the destination, and que to which it has to post the
message.
Add que_id, to differentiate between the destination
and que.
Change-Id: I94f81cb4b976dba4571f9cd0d6e059dc72024fcd
CRs-Fixed: 2308108
To help triage issues where the scheduler thread is processing slowly,
add a scheduler processing history for offline analysis. This can also
be useful in determining if a processing delay in some operation is
attributed to the scheduler thread or not.
Change-Id: Icfea17efc34b2a23d4f8b2baebe08db4e2f6ce8b
CRs-Fixed: 2307252
Currently the scheduler is unaware of the source module
which posts the message to another layer.
To trace the src, dest, and the message under ML
logging, modify current infra, to include the src,
dest in the qip argument of the API
scheduler_post_msg_by_priority
Change-Id: Ia234f60cc33fcc37ab741c462052a6e50993e3bd
CRs-Fixed: 2300537
Drop beacon/probe frames before posting on Scan queue if the queue
already has too many beacon/probes to process.
Also add scheduler API to get the queue size given the module ID.
Change-Id: I9153c7e77e74377863774b68e8163839e992358d
CRs-Fixed: 2298584
As part of the effort to reduce overly verbose and excessive logging
driver wide, reduce or remove logs in scheduler as much as possible.
Change-Id: I1e45c439315e81d9ff05e1688fe690d0ca30e097
CRs-Fixed: 2280337
QDF_DEBUG_PANIC has recently gained a reason parameter. Move instance of
logging followed immediately by QDF_DEBUG_PANIC to single calls.
Change-Id: Ifc00be410621005e8494074ed00b1b7e44cc6a03
CRs-Fixed: 2271769
Currently, the scheduler thread keeps a large, pre-allocated array of
messages for use in message posting. The vast majority of the time,
however, the scheduler thread has zero or one messages pending in the
queue. This leads to a huge memory overhead for nominal driver
operation.
Replace the current pre-allocated scheduler message pool with a
hybrid static/dynamic approach.
Change-Id: Ie942bacfef43edf142a9f35ad0309069096cda90
CRs-Fixed: 2204172
A global timer is created and freed by each module, if passed as
msg to mc queues, shouldn't be freed in scheduler_cleanup_queues,
or double free issue happens
For example, a timer in tLimTimers is freed in mac_stop->pe_stop
->lim_cleanup_mlm
Change-Id: I919b13f2570e3496a1de249fd90a88f3bfeaa131
CRs-Fixed: 2220514
Add scheduler message handler function for QDF_MODULE_ID_SCAN queue.
Also post scan beacon/probe frames and scan events to SCAN queue instead
of TARGET_IF queue.
Change-Id: I94e3ea2f0eb5c9592f5cfcd91738778fa65ba815
CRs-Fixed: 2214554
Part of the scheduler component has been migrated to the new sched_*
logging wrappers, but part of it still remains to be ported. Finish
migrating all of the scheduler logging APIs.
Change-Id: I5065d849a1523d6c27d6c6a27b1ad4605036c3e6
CRs-Fixed: 2201038
A commonly used and robust life-cycle pattern consists of the following
4 steps:
1) create (allocate resources)
2) start (mark resources as safe for use)
3) stop (mark resources as unsafe for use)
4) destroy (deallocate resources)
This pattern effectively prevents access to uninitialized and freed
resources. While the dispatcher psoc life-cycle follows this pattern,
the global life-cycle is condensed into steps 1 and 4. Unsurprisingly,
this led to uninitialized and freed resource access problems, especially
in regards to the scheduler thread.
Split the scheduler init and deinit life-cycle functions into init,
enable, disable, and deinit. Create new global enable and disable
dispatcher functions, and call the new scheduler APIs as appropriate.
This brings the global dispatcher life-cycle in line with the existing
psoc life-cycle, and prevents many scheduler related resource access
issues.
Change-Id: I58b65be0611f48e48354f28c221185f6f490f30c
CRs-Fixed: 2153283
There are many minor discrepancies in the scheduler module's init and
deinit paths. Refactor the scheduler init/deinit code paths to address
any discrepancies between the two. Also ensure error path cleanup cleans
up all the resources that were previously created.
Change-Id: Ib392f41a887031cd710aebaca32eb653952550a4
CRs-Fixed: 2153210
A set of logging helpers for the scheduler component were previously
added to reduce boilerplate logging code. Update scheduler_api.c to use
the new logging helpers.
Change-Id: I45715e40dbdd5a94d064ca4c64290a443cacea34
CRs-Fixed: 2152575
The basic qdf bit ops are not atomic. Race conditions
can lead to messages stuck in queues without mc thread
being woken up.
Change-Id: I42093b863c7ed93c5c2baf0bd14106eb4fe082fc
CRs-Fixed: 2118014
In the event of a scheduler thread watchdog bite, dump the scheduler
thread stack trace to aid in debugging.
Change-Id: I222556c7cb01985abe20b04a9383fc6c6d0724fc
CRs-Fixed: 2091875
In situations where the scheduler thread has been stopped, but some
threads are still trying to post massages to it, a particular assert
can be triggered excessively. Remove the post message assert when the
scheduler thread is stopped, and instead simply log an error.
Change-Id: I5b898dc3ea22bbedcdf2287ef6f3bcee1560bb6b
CRs-Fixed: 2046232
Changed the log levels from ERROR to DEBUG for few prints
in the scheduler API's since the prints were not for errors
but for debugging purposes.
Change-Id: I2a351de5933cfe6ec7e52d48fbc6e32880854276
CRs-Fixed: 2034873
Provide fix such a way that scheduler's initialization happens at the end after
all components' initialization gets done. Same-way scheduler's
deinitiazation happens very first before other component's deinit gets
started.
scheduler also need to call component's provided callback to notify
that component while flushing component's message queue.
CRs-Fixed: 2034772
Change-Id: I2c46bf9e68b93ecbc2ce9659d56de2887573411e
In scheduler core message processor rename camel case variable name
vStatus to qdf_status.
Change-Id: I2b22915269aad9583ab4a360b92938df3d976c66
CRs-Fixed: 2037960
In scheduler core message processor rename camel case variable name
pMsgWrapper to msg_wrapper.
Change-Id: Ica6715d9b14a807fdabc2da302de6ead4509aef8
CRs-Fixed: 2037958
In order to catch long running scheduler messages, start a watchdog
timer before processing each message, and stop the timer after each
message is processed. When the watchdog timer expires, immediately
crash the device in debug builds for easier debugging.
Change-Id: I4b43a9e12fc1f5b8a795fec790fe7548a100d9db
CRs-Fixed: 2037094
Initialize scheduler_msg on stack before posting using scheduler API
such that un-used fields in message structure are initialized to 0.
Change-Id: Ib3183b2c65eb9affd6610f4572c751c39a25fff7
CRs-Fixed: 2034772
If scheduler thread thread doesn't exist or has been shutdown then
don't post a message otherwise it will cause a memory leak.
validate the existence of scheduler thread before posting a message.
Change-Id: Id0e75bdebc7db8c00d9efb8a6e8678491b21a20f
CRs-Fixed: 2031651
Scheduler context size is more than 400KB so run time memory
allocation may fail when system memory is fragmented. Do not
allocate scheduler context memory dynamcially instead allocate
it from .bss section.
Change-Id: I19ebd0689f81b971bde271e1a540952cf41a6cba
CRs-Fixed: 2020711
Driver is missing couple of deregister handler while bringing down the
scheduler.
Change-Id: I99fe6e9a5d3cac800154945e41e707ebb7ee6396
CRs-Fixed: 2001133
By default qid to qidx mapping gets value zero as default
initialization which creates issue to identify whether zero
is valid or invalid index.
Initializing to maximum value will eliminate above mentioned
issue.
Change-Id: I4c0e2ba45e67fe6e167bd965bdb7ab901c3828a8
CRs-Fixed: 2001133
Exporting the QDF timer API's required for enabling scheduler based timers.
Fixed the compilation issues when scheduler code is enabled.
Change-Id: I148bb17c361fdff101fab3711af3c0a6c4158963
CRs-Fixed: 1117366
