PHP implements sensitive word filtering system "suggestions collection"

Hello everyone , I meet you again , I'm your friend, Quan Jun .

Code description

1、 Sensitive thesaurus maintenance update script ：

reload_dict.php, Provides automatic dictionary update to trie-tree Documentation process

PHP

<?php

//  Set memory 
ini_set('memory_limit', '128M');

//  Read sensitive word dictionary 
$handle = fopen('dict.txt', 'r');

//  Generate empty trie-tree-filter
$resTrie = trie_filter_new();

while(! feof($handle)) {
    $item = trim(fgets($handle));

    if (empty($item)) {
        continue;
    }

    //  Add sensitive words one by one trie-tree
    trie_filter_store($resTrie, $item);
}

//  Generate trie-tree file 
$blackword_tree = 'blackword.tree';

trie_filter_save($resTrie, $blackword_tree);

Copy

2、trie Tree objects get tool classes

FilterHelper.php, Provide access to trie-tree object , Avoid duplicate generation trie-tree Objects and guarantees tree File and sensitive thesaurus synchronization update

PHP

<?php
/**
 *  Filter assistant 
 *
 * getResTrie  Provide trie-tree object ;
 * getFilterWords  Extract the filtered string 
 *
 * @author W.Y.P ([email protected])
 */


class FilterHelper
{

    // trie-tree object 
    private static $_resTrie = null;
    //  Update time of dictionary tree 
    private static $_mtime = null;


    /**
     *  Prevent initialization 
     */
    private function __construct() {}


    /**
     *  Prevent cloning objects 
     */
    private function __clone() {}


    /**
     *  Provide trie-tree object 
     *
     * @param $tree_file  Dictionary tree file path 
     * @param $new_mtime  The update time of the dictionary tree at the current call 
     * @return null
     */
    static public function getResTrie($tree_file, $new_mtime) {

        if (is_null(self::$_mtime)) {
            self::$_mtime = $new_mtime;
        }

        if (($new_mtime != self::$_mtime) || is_null(self::$_resTrie)) {
            self::$_resTrie = trie_filter_load($tree_file);
            self::$_mtime = $new_mtime;

            //  Output dictionary file overload time 
            echo date('Y-m-d H:i:s') . "\tdictionary reload success!\n";
        }

        return self::$_resTrie;
    }


    /**
     *  Extract the filtered sensitive words from the original string 
     *
     * @param $str  Original string 
     * @param $res 1-3  Express   From the position 1 Start ,3 Character length 
     * @return array
     */
    static public function getFilterWords($str, $res)
    {
        $result = array();
        foreach ($res as $k => $v) {
            $word = substr($str, $v[0], $v[1]);

            if (!in_array($word, $result)) {
                $result[] = $word;
            }
        }

        return $result;
    }
}

Copy

3、 Provide external filtering HTTP Access interface

filter.php, Use swool, External submission filter interface access

PHP

<?php

//  Set the maximum running memory of the script , Adjust according to the dictionary size 
ini_set('memory_limit', '512M');

//  Set time zone 
date_default_timezone_set('Asia/Shanghai');

//  Load the helper file 
require_once('FilterHelper.php');

// http Service bound ip And port 
$serv = new swoole_http_server("182.92.177.16", 9502);


/**
 *  Processing requests 
 */
$serv->on('Request', function($request, $response) {

    //  receive get Request parameters 
    $content = isset($request->get['content']) ? $request->get['content']: '';

    $result = '';

    if (!empty($content)) {

        //  Dictionary tree file path , The default directory is 
        $tree_file = 'blackword.tree';

        //  Clear file state cache 
        clearstatcache();

        //  When getting a request , The modification time of the dictionary tree file 
        $new_mtime = filemtime($tree_file);

        //  Get the latest trie-tree object 
        $resTrie = FilterHelper::getResTrie($tree_file, $new_mtime);

        //  filtering 
        $arrRet = trie_filter_search_all($resTrie, $content);

        //  Extract the filtered sensitive words 
        $a_data = FilterHelper::getFilterWords($content, $arrRet);

        $result = json_encode($a_data);
    }

    //  Definition http Service information and response processing results 
    $response->cookie("User", "W.Y.P");
    $response->header("X-Server", "W.Y.P WebServer(Unix) (Red-Hat/Linux)");
    $response->header('Content-Type', 'Content-Type: text/html; charset=utf-8');
    $response->end($result);
});

$serv->start();

Copy

Publisher ： Full stack programmer stack length , Reprint please indicate the source ：https://javaforall.cn/130918.html Link to the original text ：https://javaforall.cn