eMMC iowait optimize

一.描述:eMMC io的问题除了硬件物料的差异软件上可优化的方向很少。本FAQ提供一些软码优化的方向供参考。二. 解决方案:1. Some mount parametersIn kernel-4.*/Documentation/filesystems/ext4.txt(1) noauto_da_alloc/auto_da_allocMany broken applications dont use fsync() when replacing existing files via patterns such asfd open(foo.new)/write(fd,..)/close(fd)/rename(foo.new, foo), or worse yet,fd open(foo, O_TRUNC)/write(fd,..)/close(fd). If auto_da_alloc is enabled, ext4 will detectthe replace-via-rename and replace-via-truncate patterns and force that any delayed allocationblocks are allocated such that at the next journal commit, in the default dataorderedmode, the data blocks of the new file are forced to disk before the rename() operation iscommitted. This provides roughly the same level of guarantees as ext3, and avoids thezero-length problem that can happen when a system crashes before the delayed allocationblocks are forced to disk.默认release代码是noauto_da_alloc即不使用延迟分配(2) barrier/nobarrierThis enables/disables the use of write barriers in the jbd code. barrier0 disables, barrier1 enables.This also requires an IO stack which can support barriers, and if jbd gets an error on a barrierwrite, it will disable again with a warning. Write barriers enforce proper on-disk orderingof journal commits, making volatile disk write caches safe to use, at some performance penalty. Ifyour disks are battery-backed in one way or another, disabling barriers may safely improve performance.The mount options barrier and nobarrier can also be used to enable or disable barriers, forconsistency with other ext4 mount options.barrier 开启write barrier(提供写顺序,记录日志,安全系数高但会影响performance)nobarrier 禁止write barrier// F2FS的文件系统mount属性通常会这样加 fsync_modenobarrier(3) .Data ModeThere are 3 different data modes:* writeback modeIn datawriteback mode, ext4 does not journal data at all. This mode providesa similar level of journaling as that of XFS, JFS, and ReiserFS in its defaultmode - metadata journaling. A crashrecovery can cause incorrect data toappear in files which were written shortly before the crash. This mode willtypically provide the best ext4 performance.* ordered modeIn dataordered mode, ext4 only officially journals metadata, but it logicallygroups metadata information related to data changes with the data blocks into asingle unit called a transaction. When its time to write the new metadataout to disk, the associated data blocks are written first. In general,this mode performs slightly slower than writeback but significantly faster than journal mode.* journal modedatajournal mode provides full data and metadata journaling. All new data iswritten to the journal first, and then to its final location.In the event of a crash, the journal can be replayed, bringing both data andmetadata into a consistent state. This mode is the slowest except when dataneeds to be read from and written to disk at the same time where itoutperforms all others modes. Enabling this mode will disable delayedallocation and O_DIRECT support.datawriteback 性能高; 写回模式先写metadata(代表日志),后写datadataordered 性能中; 命令模式先写data, 后写metadata最后写 metadata journaldatajournal 性能低; 日志模式先日志(metadata journal,data journal)后数据(metadata,data)(3) In kernel-4.*/Documentation/filesystems/f2fs.txtfsync_mode%sControl the policy of fsync. Currently supports posix, strict, and nobarrier. In posix mode, which is default, fsync will follow POSIX semantics and does a light operation to improve the filesystem performance. In strict mode, fsync will be heavy and behaves in line with xfs, ext4 and btrfs, where xfstest generic/342 will pass, but the performance will regress. nobarrier is based on posix, but doesnt issue flush command for non-atomic files likewise nobarrier mount option. (such asfsync_modenobarrier).(4) discard/nodiscardEnable/disable real-time discard in f2fs, if discard is enabled, f2fs will issue discard/TRIM commands when a segment is cleaned. Enable -o discard by default after kernel-4.19, we can refer to https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?ida39e5365835edcdb12140d423573c2b8ed39ebfb(5) flush_mergeMerge concurrent cache_flush commands as much as possible to eliminate redundant command issues. If the underlying device handles the cache_flush command relatively slowly, recommend to enable this option.如果客户要提高IO性能根据上述介绍自行评估修改这些mount属性。修改的地方是 vendor/mediatek/proprietary/hardware/fstab/mt6***/fstab.in or fstab.in.emmc 文件里的FS_FLAG_DATA_PRE。其中fstab.in.emmc for EMMC storagefstab.in for UFS storage。2. Some parameters of block layer2.1 Parameters introductioncd /proc/sys/vm/In kernel-4.9/Documentation/sysctl/vm.txtdirty_expire_centisecs -- default is 200This tunable is used to define when dirty data is old enough to be eligible for writeout by the kernel flusher threads. It is expressed in 100ths of a second. Data which has been dirty in-memory for longer than this interval will be written out next time a flusher thread wakes up.dirty_background_ratio -- default is 5It is the percentage of system memory which when dirty then system can start writing data to the disks.Note: dirty_background_bytes is the counterpart of dirty_background_ratio. Only one of them may be specified at a time, the other appears as 0 when read.dirty_ratio -- default is 20It is percentage of system memory which when dirty, the process doing writes would block and write out dirty pages to the disks.Note: dirty_ratio is the counterpart of dirty_bytes. Only one of them may bespecified at a time, the other appears as 0 when read.dirty_writeback_centisecs -- default is 300The kernel flusher threads will periodically wake up and write old data out to disk. Set to zero disables periodic writeback altogether.2.2 通常情况下系统在如下条件触发回写a.定时回写回写线程只回写在内存中dirty时间超过(dirty_expire_centisecs/100)秒的页, 默认dirty_writeback_centisecs的值是300,dirty_expire_centisecs是200即3秒启动一次回写线程,把dirty时间超过2秒的页回写,只回写超时的dirty页.b.条件回写如果发现系统中的脏数据大于这阈值dirty_background_ratio或dirty_background_bytes 而小于dirty_ratio会触发进程去写脏数据但是用户的write调用会立即返回无需等待write系统调用不会被阻塞。如果发现系统中的脏数据大于这阈值dirty_ratio或dirty_bytes 需要自己把脏数据刷回磁盘直到降低到这个阈值以下才返回write系统调用会被阻塞。c.内存不足的时候脏页回写,回收memory,直到空闲页面满足需求为止。2.3 参数修改建议客户可以根据 FAQ13684 减小 dirty_background_ratio、dirty_ratio [FAQ13684] iowait问题成因及对策可以在device/mediatek/mt6***/init.mt6***.rc增加FAQ中的修改只要保证添加的位置能够执行到即可。修改是否成功可以在开机后进入adb shell用下面命令查看rootandroid:/ # cat /proc/sys/vm/dirty_ratiorootandroid:/ # cat /proc/sys/vm/dirty_background_ratio3. 平台端的优化3.1 安装APP的时候拉CPU/DDR频率In vendor/mediatek/proprietary/hardware/power/config/mt67**/scn_tbl/powerscntbl.cfgAndroid O/P usedMTK_POWER_HINT_PMS_INSTALLCMD_SET_CLUSTER_CPU_CORE_MIN, MTK_POWER_HINT_PMS_INSTALL, 0, 4CMD_SET_CLUSTER_CPU_FREQ_MIN, MTK_POWER_HINT_PMS_INSTALL, 0, 3000000CMD_SET_CLUSTER_CPU_CORE_MIN, MTK_POWER_HINT_PMS_INSTALL, 1, 4CMD_SET_CLUSTER_CPU_FREQ_MIN, MTK_POWER_HINT_PMS_INSTALL, 1, 3000000CMD_SET_OPP_DDR, MTK_POWER_HINT_PMS_INSTALL, 0Android Q/R usedMTKPOWER_HINT_PMS_INSTALLscenario powerhintMTKPOWER_HINT_PMS_INSTALLdata cmdPERF_RES_CPUFREQ_MIN_CLUSTER_0 param13000000/datadata cmdPERF_RES_CPUFREQ_MIN_CLUSTER_1 param13000000/datadata cmdPERF_RES_DRAM_OPP_MIN param10/data/scenario3.2 read_ahead size通过数据预读并且记载到随机访问内存方式提高磁盘读操作. 值太小起不到效果值太大会导致读出很多无用的数据而影响performanceIn device/mediatek/mt67**/init.mt67**.rc# end boot time fs tuneon property:sys.boot_completed1write /sys/block/mmcblk0/queue/iostats 1write /sys/block/mmcblk0/queue/read_ahead_kb 128write /sys/block/mmcblk0/queue/nr_requests 128write /sys/block/dm-0/queue/read_ahead_kb 128write /sys/block/dm-1/queue/read_ahead_kb 128write /sys/block/dm-2/queue/read_ahead_kb 128// memory size 小于等于 2GB的默认推荐用 128KB// memory size 大于等于 4GB的默认推荐用 512KB//UFS/DDR一体的memory基本比较大推荐512KB如 write /sys/block/sdc/queue/read_ahead_kb 5123.3 提高 EMMC IO thread的优先级a. kernel 4.4/4.9/4.14kernel-4.*/drivers/mmc/core/core.cint mmc_run_queue_thread(void *data){struct mmc_host *host data;struct mmc_request *cmd_mrq NULL;struct mmc_request *dat_mrq NULL;struct mmc_request *done_mrq NULL;unsigned int task_id, areq_cnt_chk, tmo;bool is_err false;bool is_done false;int err;u64 chk_time 0;struct sched_param scheduler_params {0};/* Set as RT priority */scheduler_params.sched_priority 1;sched_setscheduler(current, SCHED_FIFO, scheduler_params); // 提高IO thread 优先级kernel-4.*/drivers/mmc/card/queue.cstatic int mmc_queue_thread(void *d){struct mmc_queue *mq d;struct request_queue *q mq-queue;#ifdef CONFIG_MTK_EMMC_CQ_SUPPORTint cmdq_full 0;unsigned int tmo;#endifbool part_cmdq_en false;struct sched_param scheduler_params {0};scheduler_params.sched_priority 1;sched_setscheduler(current, SCHED_FIFO, scheduler_params); // 提高IO thread 优先级b. kernel 4.19因为R0版本kernel版本升级到kernel-4.19架构变动大R0 6761/6762/6765的EMMC thread提高为RT优先级后可能出现hang detect的问题。如果hang住需要回退patch ALPS05897129部分版本已经回退了。patch ALPS05897129说明如下:Refer to ALPS05897129, If it only sets exe_cq as RT thread, areq_cnt also needs to change together with enqueue flow, or KE will happen, so i dot not recommend to set MMC RT thread on kernel-4.19 at present.如果要提高kernel-4.19 MMC thread优先级, 可以用set_user_nice(), 建议不要超过100比如下面设置优先级110的diff --git a/kernel-4.19/drivers/mmc/core/core.c b/kernel-4.19/drivers/mmc/core/core.cindex 2187072..642173f 100644--- a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c -895,6 896,11 int mmc_run_queue_thread(void *data)bool is_done false;int err;u64 chk_time 0; //up the emmc cmdq thread priority to 110 (nice-10) set_user_nice(current, -10);pr_info([CQ] start cmdq thread\n);mt_bio_queue_alloc(current, NULL, false);