License Plate Detection Application

Overview

LicensePlate_Project ๐Ÿš— ๐Ÿš™

[Project] 2021.02 ~ 2021.09 License Plate Detection Application

Overview


1. ๋ฐ์ดํ„ฐ ์ˆ˜์ง‘ ๋ฐ ๋ผ๋ฒจ๋ง

์ฐจ๋Ÿ‰ ๋ฒˆํ˜ธํŒ ์ด๋ฏธ์ง€๋ฅผ ์ง์ ‘ ์ˆ˜์ง‘ํ•˜์—ฌ ๊ฐ ์ด๋ฏธ์ง€์— ๋Œ€ํ•ด '๋ฒˆํ˜ธํŒ ๊ธ€์ž'์™€ '๋ฒˆํ˜ธํŒ ๋„ค ๊ผญ์ง“์ ์˜ x,y ์ขŒํ‘œ'๋ฅผ ๋ผ๋ฒจ๋ง ํ•œ๋‹ค.

๋ฒˆํ˜ธํŒ ์ด๋ฏธ์ง€
๋ผ๋ฒจ๋ง 20210210_222919.jpg 1481 2773 2043 2689 2043 2794 1486 2883 36์กฐ 2428

ํ…์ŠคํŠธ ํŒŒ์ผ๋กœ ์ €์žฅ๋œ ๋ผ๋ฒจ๋ง ์ •๋ณด๋Š” ๋ฒˆํ˜ธํŒ ๋„ค ๊ผญ์ง“์ ์˜ ์ ˆ๋Œ€ ์ขŒํ‘œ์™€ ๋ฒˆํ˜ธํŒ ๊ธ€์ž๋ฅผ ํฌํ•จํ•˜๊ณ  ์žˆ๋‹ค. ํ•™์Šต ๋ฐ์ดํ„ฐ์˜ 20%๋ฅผ ๊ฒ€์ฆ ๋ฐ์ดํ„ฐ๋กœ ๋‚˜๋ˆ„์–ด ๋ฐ์ดํ„ฐ์…‹ ์ค€๋น„๋ฅผ ๋งˆ์นœ๋‹ค. ์ตœ์ข… ๋ฐ์ดํ„ฐ์…‹ ๊ตฌ์„ฑ์€ ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค.

ํ•™์Šต ๋ฐ์ดํ„ฐ ๊ฒ€์ฆ ๋ฐ์ดํ„ฐ
1635์žฅ 409์žฅ

2. YOLOv5 ํ•™์Šต (Pytorch-YOLOv5)

  • ์ฐธ๊ณ : https://github.com/ultralytics/yolov5

  • ์ธํ’‹ ๋ฐ์ดํ„ฐ ์ค€๋น„
    ์›๋ณธ ์ด๋ฏธ์ง€๋Š” ๋ฒˆํ˜ธํŒ ์˜์—ญ์„ ํƒ์ง€ํ•˜๊ธฐ ์œ„ํ•ด ๊ณง์žฅ YOLO์˜ ์ž…๋ ฅ์œผ๋กœ ์‚ฌ์šฉ๋˜๊ธฐ ๋•Œ๋ฌธ์—, YOLO์˜ ์ž…๋ ฅ ํ˜•์‹์— ๋งž์ถ”๊ธฐ ์œ„ํ•ด ๊ฐ ์ด๋ฏธ์ง€ ๋งˆ๋‹ค ์ด๋ฏธ์ง€ ํŒŒ์ผ๋ช…๊ณผ ๋™์ผํ•œ ์ด๋ฆ„์˜ ํ…์ŠคํŠธ ํŒŒ์ผ์„ ๋งŒ๋“ค์–ด bounding box์˜ ์ขŒํ‘œ ์ •๋ณด๋ฅผ class, x_center, y_center, width, height์˜ ํฌ๋งท์˜ ๋ฌธ์ž์—ด๋กœ ์ €์žฅํ•œ๋‹ค. ์ด ๋•Œ, class๋ฅผ ์ œ์™ธํ•œ ๋‚˜๋จธ์ง€ ๊ฐ’์€ ๋ชจ๋‘ 0-1 ์‚ฌ์ด์˜ ์ƒ๋Œ€ ์ขŒํ‘œ๋กœ ๋ณ€ํ™˜ํ•œ๋‹ค.

โ”œโ”€โ”€ Yolo_input
    โ”œโ”€โ”€ train
    โ”‚   โ”œโ”€โ”€ images
    โ”‚   โ”‚   โ”œโ”€โ”€ 1.jpg
    โ”‚ 	โ”‚   โ”œโ”€โ”€ 2.jpg
    โ”‚ 	โ”‚  	โ”‚     :
    โ”‚ 	โ”‚  		  
    โ”‚   โ”œโ”€โ”€ labels
    โ”‚	    โ”œโ”€โ”€ 1.txt
    โ”‚	    โ”œโ”€โ”€ 2.txt
    โ”‚	   	โ”‚     :
    โ”‚	
    โ””โ”€โ”€ val
 	    โ”œโ”€โ”€ images
 	    โ”œโ”€โ”€ labels
  • dataset.yaml ์ค€๋น„
    Custom ๋ฐ์ดํ„ฐ์…‹์— YOLOv5 ํ•™์Šต ์ฝ”๋“œ๋ฅผ ๊ทธ๋Œ€๋กœ ์“ธ ๊ฒƒ์ด๊ธฐ ๋•Œ๋ฌธ์—, ๋ฐ์ดํ„ฐ์…‹ ์„ธํŒ… ๋ถ€๋ถ„๋งŒ ์ˆ˜์ •ํ•œ๋‹ค. dataset.yaml ํŒŒ์ผ์— ํ•™์Šต, ๊ฒ€์ฆ ๋ฐ์ดํ„ฐ ๊ฒฝ๋กœ์™€ ๊ฐ์ฒด ํด๋ž˜์Šค ์ •๋ณด๋ฅผ ๊ธฐ์ž…ํ•œ๋‹ค. ์šฐ๋ฆฌ ํ”„๋กœ์ ํŠธ์˜ ๊ฒฝ์šฐ ํƒ์ง€ํ•˜๋Š” ๊ฐ์ฒด๊ฐ€ ์ฐจ๋Ÿ‰ ๋ฒˆํ˜ธํŒ ํ•˜๋‚˜์ด๋ฏ€๋กœ ํด๋ž˜์Šค ๋ผ๋ฒจ์„ 0์œผ๋กœ, ์ด๋ฆ„์„ 'plate' ๋กœ ํ•œ๋‹ค.

  • YOLO ๋ชจ๋ธ ์„ ํƒ
    ๋ณธ ํ”„๋กœ์ ํŠธ๋ฅผ ์œ„ํ•ด ๊ฐ€์žฅ ์ž‘๊ณ  ๋น ๋ฅธ ๋ชจ๋ธ์ธ YOLOv5s๋ฅผ ์‚ฌ์šฉํ•˜์˜€๋‹ค.


3. ๊ผญ์ง“์  ์˜ˆ์ธก ๋ชจ๋ธ ํ•™์Šต

  • ์‚ฌ์šฉํ•œ ๋ชจ๋ธ : timm์œผ๋กœ ์‚ฌ์ „ํ•™์Šต๋œ Resnet18 ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•˜์˜€๋‹ค

  • ์ฒซ ๋ฒˆ์งธ ๋ฐฉ๋ฒ•

    1. ์‚ฌ์šฉ๋œ ์ด๋ฏธ์ง€ : ๋„ค ๊ผญ์ง“์  ์ขŒํ‘œ๊ฐ’์„ ์ด์šฉํ•˜์—ฌ ๋งŒ๋“  ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค์—์„œ ๊ฐ ์ถ•์œผ๋กœ 1%์”ฉ ๋Š˜์ธ ์ด๋ฏธ์ง€

    2. ๋ฐ์ดํ„ฐ ์ฆ๊ฐ•: ์ „๋‹จ ๋ณ€ํ™˜(shear transformation), ์‚ฌ์ง„ํ•ฉ์„ฑ, ๋ฐ๊ธฐ์กฐ์ ˆ, ๋ฆฌ์‚ฌ์ด์ฆˆ
      ์ž…๋ ฅ ์ด๋ฏธ์ง€๋ฅผ ์ „๋‹จ ๋ณ€ํ™˜ ๊ธฐ๋ฒ•์„ ์ด์šฉํ•ด x, y์ถ•์œผ๋กœ ๋žœ๋คํ•˜๊ฒŒ ๋ณ€ํ™˜ํ•˜๋ฉด ๊ฒ€์€์ƒ‰ ์—ฌ๋ฐฑ ๋ถ€๋ถ„์ด ์ƒ๊ฒจ, ์ด ๋ถ€๋ถ„์„ ๋‹ค๋ฅธ ์ด๋ฏธ์ง€์—์„œ ๋žœ๋คํ•˜๊ฒŒ ๊ฐ€์ ธ์™€ ํ•ฉ์„ฑ์‹œ์ผฐ๋‹ค. ์ด ์ด๋ฏธ์ง€์— ๋žœ๋ค์œผ๋กœ ๋ฐ๊ธฐ์กฐ์ ˆ์„ ์ถ”๊ฐ€ํ•˜์—ฌ, 128x128 ์ด๋ฏธ์ง€๋กœ ๋ฆฌ์‚ฌ์ด์ฆˆํ•œ ์ด๋ฏธ์ง€๋ฅผ ๋ชจ๋ธ์— ์ž…๋ ฅ์œผ๋กœ ๋„ฃ์—ˆ๋‹ค.

    3. ๋ฌธ์ œ์  : ๊ฒ€์€์ƒ‰ ๋ถ€๋ถ„์„ ๋‹ค๋ฅธ ์‚ฌ์ง„์œผ๋กœ ํ•ฉ์„ฑ์‹œ์ผฐ๋”๋‹ˆ ์‹ค์„ธ๊ณ„ ๋ฐ์ดํ„ฐ์™€ ๊ดด๋ฆฌ๊ฐ์ด ์ƒ๊ฒจ ์„ฑ๋Šฅ ์ €ํ•˜ ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•˜์˜€๋‹ค.

    ์‚ฌ์šฉ๋œ ์ด๋ฏธ์ง€ ๋ฐ์ดํ„ฐ์ฆ๊ฐ•1 ๋ฐ์ดํ„ฐ์ฆ๊ฐ•2
    ์‚ฌ์šฉ๋œ ์ด๋ฏธ์ง€ ๋ฐ์ดํ„ฐ์ฆ๊ฐ•1 ๋ฐ์ดํ„ฐ์ฆ๊ฐ•2
  • ๋‘ ๋ฒˆ์งธ ๋ฐฉ๋ฒ•

    1. ์‚ฌ์šฉ๋œ ์ด๋ฏธ์ง€ : ์›๋ณธ ์ด๋ฏธ์ง€

    2. ๋ฐ์ดํ„ฐ ์ฆ๊ฐ•: ์ „๋‹จ ๋ณ€ํ™˜, ๋ฐ๊ธฐ์กฐ์ ˆ, ๋ฆฌ์‚ฌ์ด์ฆˆ ์ž…๋ ฅ ์ด๋ฏธ์ง€์™€ ๋ผ๋ฒจ๋ง์„ ํ†ตํ•ด ์•Œ๋ ค์ง„ ๋ฒˆํ˜ธํŒ ๊ผญ์ง“์ ์˜ ์ขŒํ‘œ๋“ค์„ ์ „๋‹จ ๋ณ€ํ™˜ ๊ธฐ๋ฒ•์„ ์ด์šฉํ•ด ๋žœ๋ค ๊ฐ’์œผ๋กœ ๋ณ€ํ™˜ํ•œ๋‹ค. ์ด ์ด๋ฏธ์ง€์—์„œ ๋ฒˆํ˜ธํŒ์˜ ์ขŒํ‘œ๋ฅผ ๊ธฐ์ค€์œผ๋กœ margin์„ ์ฃผ๊ณ , ๊ทธ ์ง€์ ์œผ๋กœ๋ถ€ํ„ฐ ๋žœ๋คํ•˜๊ฒŒ ์ขŒํ‘œ๋ฅผ ์ฐ์–ด ์ด๋ฏธ์ง€๋ฅผ ์ž๋ฅธ ๊ฒƒ์„ ์‚ฌ์šฉ. ์ด๋ ‡๊ฒŒ ํ•˜๋ฉด ์ฒซ ๋ฒˆ์งธ ๋ฐฉ๋ฒ•์—์„œ ๋‚˜์™”๋˜ ๊ฒ€์€ ์—ฌ๋ฐฑ ๋ถ€๋ถ„์ด ๋‚˜์˜ค์ง€ ์•Š์œผ๋ฏ€๋กœ ์‹ค์„ธ๊ณ„ ๋ฐ์ดํ„ฐ์™€ ๋” ๊ทผ์ ‘ํ•˜๋‹ค. ์ด ์ด๋ฏธ์ง€์— ๋žœ๋ค์œผ๋กœ ๋ฐ๊ธฐ์กฐ์ ˆ์„ ์ถ”๊ฐ€ํ•˜์—ฌ, 128x128 ์ด๋ฏธ์ง€๋กœ ๋ฆฌ์‚ฌ์ด์ฆˆํ•œ ์ด๋ฏธ์ง€๋ฅผ ๋ชจ๋ธ์— ์ž…๋ ฅ์œผ๋กœ ๋„ฃ์—ˆ๋‹ค.

    ์‚ฌ์šฉ๋œ ์ด๋ฏธ์ง€ ๋ฐ์ดํ„ฐ์ฆ๊ฐ•1 ๋ฐ์ดํ„ฐ์ฆ๊ฐ•2
    ์‚ฌ์šฉ๋œ ์ด๋ฏธ์ง€ ๋ฐ์ดํ„ฐ์ฆ๊ฐ•1 ๋ฐ์ดํ„ฐ์ฆ๊ฐ•2
  • Output : ์ƒํ•˜์ขŒ์šฐ ๋„ค ๊ผญ์ง“์ ์— ๋Œ€ํ•œ X,Y ์ƒ๋Œ€์ขŒํ‘œ


4. ๊ธ€์ž ์˜ˆ์ธก ๋ชจ๋ธ ํ•™์Šต

  • ์‚ฌ์šฉํ•œ ๋ชจ๋ธ : timm์œผ๋กœ ์‚ฌ์ „ํ•™์Šต๋œ Resnet18 ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•˜์˜€๋‹ค.

  • ์ฒซ ๋ฒˆ์งธ ๋ฐฉ๋ฒ•

    1. ์‚ฌ์šฉ๋œ ์ด๋ฏธ์ง€ : ์›๋ณธ ์ด๋ฏธ์ง€์˜ ๋„ค ๊ผญ์ง“์  ์ขŒํ‘œ์— ๋Œ€ํ•œ ground truth๋ฅผ ์ด์šฉํ•˜์—ฌ (128, 256)์˜ ํฌ๊ธฐ๋กœ ํˆฌ์˜๋ณ€ํ™˜ํ•œ ์ด๋ฏธ์ง€

    2. ๋ฐ์ดํ„ฐ ์ฆ๊ฐ•: Salt & Pepper ๋…ธ์ด์ฆˆ ์‹ค์ œ ์ฐจ๋Ÿ‰์˜ ๋ฒˆํ˜ธํŒ์€ ๋จผ์ง€ ๋ฐ ๋ฒŒ๋ ˆ์™€ ๊ฐ™์€ ์ด๋ฌผ์งˆ ๋•Œ๋ฌธ์— ์–ผ๋ฃฉ๋œ๋ฃฉํ•œ ๊ฒฝ์šฐ๊ฐ€ ๋งŽ๋‹ค. ๋”ฐ๋ผ์„œ ์ž…๋ ฅ ๋ฐ์ดํ„ฐ์— ๋žœ๋คํ•œ ๋…ธ์ด์ฆˆ๋ฅผ ์ถ”๊ฐ€ํ•˜์—ฌ ์ผ๋ฐ˜์ ์ธ ์ƒํ™ฉ๊นŒ์ง€ ์ปค๋ฒ„ํ•  ์ˆ˜ ์žˆ๋„๋ก ํ•˜์˜€๋‹ค.

    3. ๋ฌธ์ œ์  : ์‹ค์ œ ์ถ”๋ก  ๊ณผ์ •์—์„œ๋Š” ๊ผญ์ง“์  ์˜ˆ์ธก ๋ชจ๋ธ๋กœ๋ถ€ํ„ฐ ์˜ˆ์ธก๋œ ๊ผญ์ง“์  ๊ฐ’์„ ๊ธฐ๋ฐ˜์œผ๋กœ ์ •๋ ฌ๋œ ๋ฒˆํ˜ธํŒ ์ด๋ฏธ์ง€๊ฐ€ ์ž…๋ ฅ์œผ๋กœ ์‚ฌ์šฉ๋˜๋ฏ€๋กœ, ๊ธ€์ž ์˜ˆ์ธก ๋ชจ๋ธ์˜ ์„ฑ๋Šฅ์ด ๊ผญ์ง“์  ์˜ˆ์ธก ๋ชจ๋ธ์˜ ์„ฑ๋Šฅ์— ํฐ ์˜ํ–ฅ์„ ๋ฐ›์Œ์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ์—ˆ๋‹ค.

  • ๋‘ ๋ฒˆ์งธ ๋ฐฉ๋ฒ•

    1. ์‚ฌ์šฉ๋œ ์ด๋ฏธ์ง€ : ์›๋ณธ ์ด๋ฏธ์ง€์˜ ๋„ค ๊ผญ์ง“์  ์ขŒํ‘œ๋ฅผ x,y ๋ฐฉํ–ฅ์œผ๋กœ ๊ฐ๊ฐ ๋žœ๋คํ•˜๊ฒŒ ์ด๋™์‹œํ‚จ ํ›„ (128, 256)์˜ ํฌ๊ธฐ๋กœ ํˆฌ์˜๋ณ€ํ™˜ํ•œ ์ด๋ฏธ์ง€

    2. ๋ฐ์ดํ„ฐ ์ฆ๊ฐ•: Salt & Pepper ๋…ธ์ด์ฆˆ, ๋ฐ๊ธฐ ์กฐ์ ˆ(์ „์ฒด ๋ฐ๊ฒŒ, ์ „์ฒด ์–ด๋‘ก๊ฒŒ, ๊ทธ๋ฆผ์ž) ์ˆ˜์ง‘๋œ ๋ฐ์ดํ„ฐ์…‹์€ ๋Œ€๋ถ€๋ถ„ ๋‚ฎ์— ์ฐ์€ ๋ฒˆํ˜ธํŒ ์ด๋ฏธ์ง€์˜€๊ธฐ ๋•Œ๋ฌธ์—, ํ…Œ์ŠคํŠธ ๋ฆฌํฌํŒ… ์‹œ ์•ผ๊ฐ„ ์ด๋ฏธ์ง€์— ๋Œ€ํ•ด์„œ๋Š” ์„ฑ๋Šฅ์ด ๋‚ฎ์•„์ง€๋Š” ๊ฒƒ์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ์—ˆ๋‹ค. ๋”ฐ๋ผ์„œ ๋ฐ๊ธฐ ์กฐ์ ˆ ๋ฐ ๊ทธ๋ฆผ์ž ์ถ”๊ฐ€ ์ฆ๊ฐ• ๊ธฐ๋ฒ•์„ ์ถ”๊ฐ€ํ•˜์—ฌ ์—ฌ๋Ÿฌ ํ™˜๊ฒฝ์˜ ์ด๋ฏธ์ง€์— ๋Œ€ํ•ด ๊ฐ•๊ฑดํ•œ ์„ฑ๋Šฅ์„ ๋ณด์ด๋„๋ก ํ•˜์˜€๋‹ค.

    ์ถ”๋ก  ์‹œ ์‹ค์ œ ์ž…๋ ฅ๋˜๋Š” ์ด๋ฏธ์ง€ ์ฒซ ๋ฒˆ์งธ ๋ฐฉ๋ฒ• ๋‘ ๋ฒˆ์งธ ๋ฐฉ๋ฒ•

    ๋ฐ์ดํ„ฐ ์ฆ๊ฐ•์˜ ์˜ˆ์‹œ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค.

    ์‚ฌ์šฉ๋œ ์ด๋ฏธ์ง€ ๋ฐ์ดํ„ฐ์ฆ๊ฐ•1 ๋ฐ์ดํ„ฐ์ฆ๊ฐ•2
  • Output : (๋ฐฐ์น˜์‚ฌ์ด์ฆˆ, 7, 45, 1) ๋ชจ์–‘์˜ ํ…์„œ
    7 -> 7๊ธ€์ž 45 -> 45๊ฐœ์˜ ๊ฐ€๋Šฅํ•œ ๋ฌธ์ž (['๊ฐ€', '๋‚˜', '๋‹ค', '๋ผ', '๋งˆ', '๊ฑฐ', '๋„ˆ', '๋”', '๋Ÿฌ', '๋จธ', '๋ฒ„', '์„œ', '์–ด', '์ €', '๊ณ ', '๋…ธ', '๋„', '๋กœ', '๋ชจ', '๋ณด', '์†Œ', '์˜ค', '์กฐ', '๊ตฌ', '๋ˆ„', '๋‘', '๋ฃจ', '๋ฌด', '๋ถ€', '์ˆ˜', '์šฐ', '์ฃผ', 'ํ—ˆ', 'ํ•˜', 'ํ˜ธ', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9'])


5. pt >> onnx >> pb >> tflite ๋ณ€ํ™˜

  • YOLOv5
    ์ œ๊ณตํ•ด์ฃผ๋Š” export.py๋ฅผ ์‚ฌ์šฉํ•ด TensorFlow Lite ํŒŒ์ผ๋กœ ๋ณ€ํ™˜ํ•œ๋‹ค. ์ด ๋•Œ, Non Max Suppression ๋ถ€๋ถ„์€ TensorFlow Lite๋กœ ๋ณ€ํ™˜๋˜์ง€ ์•Š์•„ ์•ˆ๋“œ๋กœ์ด๋“œ ์ŠคํŠœ๋””์˜ค ์ฝ”๋“œ๋ฅผ ์งค ๋•Œ ๋”ฐ๋กœ ์ถ”๊ฐ€ํ•˜์˜€๋‹ค. YOLO์˜ ์ถœ๋ ฅ์œผ๋กœ ๋‚˜์˜ค๋Š” (1, 3024, 6)์˜ ํ…์„œ๋Š” 3024๊ฐœ์˜ ๊ฐ€๋Šฅํ•œ ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค์™€, ๊ฐ ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค์˜ x_center, y_center, width, height, confidence, ๊ฐ์ฒด ํด๋ž˜์Šค ์ •๋ณด๋ฅผ ํฌํ•จํ•˜๊ณ  ์žˆ๋‹ค. ์•„๋ž˜ ์ฝ”๋“œ๋Š” ๊ฐ€๋Šฅํ•œ 3024๊ฐœ์˜ ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค ์ค‘ ๊ฐ€์žฅ ํฐ confidence ๊ฐ’์„ ๊ฐ€์ง€๋Š” ํ•˜๋‚˜์˜ ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค๋งŒ์„ ์ถ”๋ก ์˜ ๊ฒฐ๊ณผ๋กœ ๋งŒ๋“œ๋Š” ์ฝ”๋“œ์ด๋‹ค (Non Max Suppression).
float max_conf = detectionResult[0][0][4];
        int idx = 0;
        for(int i = 0; i<3024; i++){
            if(max_conf < detectionResult[0][i][4]){
                max_conf = detectionResult[0][i][4];
                idx = i;
            }
        }
  • ๊ผญ์ง“์  ์˜ˆ์ธก ๋ชจ๋ธ & ๊ธ€์ž ์˜ˆ์ธก ๋ชจ๋ธ
    ๋ชจ๋ธ ํ•™์Šต ์‹œ, ๊ฒ€์ฆ ๋ฐ์ดํ„ฐ์…‹์— ๋Œ€ํ•ด ๊ฐ€์žฅ ๋†’์€ ์ •ํ™•๋„๋ฅผ ๊ฐ€์ง€๋Š” ๋ชจ๋ธ์˜ ๊ฐ€์ค‘์น˜๋ฅผ onnx ํŒŒ์ผ๋กœ ์ €์žฅํ•˜๊ณ , tflite_converter.py๋ฅผ ํ†ตํ•ด ์ตœ์ข…์ ์œผ๋กœ ์–ดํ”Œ๋ฆฌ์ผ€์ด์…˜ ์ƒ์—์„œ ๋ชจ๋ธ์„ ๋กœ๋“œํ•  ๋•Œ ์“ฐ์ด๋Š” TensorFlow Lite ํŒŒ์ผ๋กœ ๋ณ€ํ™˜ํ•œ๋‹ค.

6. ์•ˆ๋“œ๋กœ์ด๋“œ ์–ดํ”Œ๋ฆฌ์ผ€์ด์…˜ ์ œ์ž‘

์–ดํ”Œ๋ฆฌ์ผ€์ด์…˜์— ์•ž์„œ ๋งŒ๋“  ํ•™์Šต๋œ ๋ชจ๋ธ๋“ค์„ ์‚ฌ์šฉํ•˜๊ธฐ ์œ„ํ•ด์„œ๋Š” ๊ฐ ๋ชจ๋ธ์— ๋Œ€ํ•œ ์ถ”๋ก  ์ฝ”๋“œ๋ฅผ ๋งŒ๋“ค๊ณ , ์ด๋ฅผ ์•ˆ๋“œ๋กœ์ด๋“œ ์ŠคํŠœ๋””์˜ค์˜ MainActivity์— ๋ถˆ๋Ÿฌ์™€์„œ ์‚ฌ์šฉํ•ด์•ผ ํ•œ๋‹ค. ์šฐ๋ฆฌ๋Š” YOLOv5(DHDetectionModel.java), ๊ผญ์ง“์  ์˜ˆ์ธก(AlignmentModel.java), ๊ธ€์ž์˜ˆ์ธก(CharModel.java) ์ด ์„ธ ๊ฐ€์ง€ ๋ชจ๋ธ์— ๋Œ€ํ•œ ์ถ”๋ก  ์ฝ”๋“œ๋ฅผ ๋งŒ๋“ค์—ˆ๋‹ค. ์ถ”๋ก  ์ฝ”๋“œ์— ์‚ฌ์šฉ๋œ ๋ฉ”์†Œ๋“œ๋“ค์€ ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค:

  • ์ƒ์„ฑ์ž

    DHDetectionModel(Activity activity, Interpreter.Options options)
    AlignmentModel(Activity activity, Interpreter.Options options)
    CharModel(Activity activity, Interpreter.Options options)

    --> ๊ฐ ์ถ”๋ก  ์ธ์Šคํ„ด์Šค๋ฅผ ์ƒ์„ฑํ•  ๋•Œ, ๋ชจ๋ธ ์ธํ„ฐํ”„๋ฆฌํ„ฐ(mInterpreter)์™€ ๋ชจ๋ธ์— ๋“ค์–ด๊ฐ€๋Š” ์ž…๋ ฅ(mImageData)์— ๋Œ€ํ•ด์„œ ์ •์˜ํ•œ๋‹ค.

  • ๊ณตํ†ต์ ์œผ๋กœ ์‚ฌ์šฉ๋œ ๋ฉ”์†Œ๋“œ

    MappedByteBuffer loadModelFile(Activity activity)

    --> tflite ํŒŒ์ผ์„ ๋ถˆ๋Ÿฌ์˜ค๋Š” ๋ฉ”์†Œ๋“œ๋กœ ์ธํ„ฐํ”„๋ฆฌํ„ฐ ์ƒ์„ฑ์‹œ์— ์‚ฌ์šฉ๋œ๋‹ค.

    void convertBitmapToByteBuffer(Bitmap bitmap)

    --> ์ถ”๋ก ํ• ๋•Œ ์ด๋ฏธ์ง€๋ฅผ ๋ชจ๋ธ์— ๋“ค์–ด๊ฐ€๋Š” ์ž…๋ ฅ ํ˜•์‹์ธ ByteBuffer์˜ ํ˜•ํƒœ๋กœ ๋ฐ”๊พธ์–ด์ฃผ๋Š” ๋ฉ”์†Œ๋“œ์ด๋‹ค.

  • ์ถ”๋ก  ๋ฉ”์†Œ๋“œ

    • DHDetectionModel

      float[][] getProposal(Bitmap bm, Mat input)

      --> ์ด๋ฏธ์ง€๊ฐ€ ์ž…๋ ฅ์œผ๋กœ ๋“ค์–ด๊ฐ€๋ฉด float[2][5] ํ˜•ํƒœ์˜ ์ •๋ณด๋ฅผ ์ถœ๋ ฅํ•œ๋‹ค. ์ถœ๋ ฅ๊ฐ’์—๋Š” ๋ชจ๋ธ์ด ํƒ์ง€ํ•œ bounding box์˜ x, y, w, h, confidence์— ๋Œ€ํ•œ ์ •๋ณด๋ฅผ ๋‹ด๊ณ  ์žˆ๋‹ค. Yolov5์— nms๊ฐ€ tflite ํ˜•ํƒœ๋กœ ๋ณ€ํ™˜๋˜์ง€ ์•Š๊ธฐ ๋•Œ๋ฌธ์— ๋”ฐ๋กœ nms ์ฝ”๋“œ๋ฅผ ์ถ”๊ฐ€ํ•˜์˜€๋‹ค.

    • AlignmentModel

      float[] getCoordinate(Bitmap bitmap)

      --> DHDetectionModel์—์„œ ๋‚˜์˜จ ์ถœ๋ ฅ์„ ์ด์šฉํ•ด bounding box์˜ ํฌ๊ธฐ๋กœ ์ž๋ฅธ ์ด๋ฏธ์ง€๊ฐ€ ์ž…๋ ฅ์œผ๋กœ ๋“ค์–ด๊ฐ€๋ฉด, float[8] ํ˜•ํƒœ์˜ ์ •๋ณด๋ฅผ ์ถœ๋ ฅํ•œ๋‹ค. ์ถœ๋ ฅ๊ฐ’์—๋Š” ๋ชจ๋ธ์ด ์˜ˆ์ธกํ•œ ๊ผญ์ง“์ ์˜ ๋„ค ์ขŒํ‘œ์˜ (x,y)๊ฐ’์„ ๋‹ด๊ณ ์žˆ๋‹ค.

    • CharModel

      String getString(Bitmap bm)

      --> AlignmentModel์—์„œ ๋‚˜์˜จ ์ถœ๋ ฅ์„ ์ด์šฉํ•ด ๋ฒˆํ˜ธํŒ ํฌ๊ธฐ๋กœ ์ด๋ฏธ์ง€๋ฅผ ์ž๋ฅธ ํ›„ ์ „๋‹จ๋ณ€ํ™˜์„ ์ด์šฉํ•ด ์ •๋ฉด์œผ๋กœ ๊ณง๊ฒŒ ํŽธ ์ด๋ฏธ์ง€๊ฐ€ ์ž…๋ ฅ์œผ๋กœ ๋“ค์–ด๊ฐ€๋ฉด, String ํ˜•ํƒœ์˜ ์ •๋ณด๋ฅผ ์ถœ๋ ฅํ•œ๋‹ค. ์ถœ๋ ฅ๊ฐ’์—๋Š” ๋ชจ๋ธ์ด ์˜ˆ์ธกํ•œ ๋ฒˆํ˜ธํŒ์˜ ๊ธ€์ž ์ •๋ณด๋ฅผ ๋‹ด๊ณ ์žˆ๋‹ค.

  • ์ถ”๋ก  ์†๋„(FPS) ๋ฌธ์ œ ๊ฐœ์„ 
    ์ดˆ๊ธฐ์— ๋ชจ๋“  ๋ชจ๋ธ๋“ค์„ ์•ฑ์— ์ ์šฉํ•˜์˜€์„ ๋•Œ, ํ•œ ์ด๋ฏธ์ง€๋ฅผ ์ฒ˜๋ฆฌํ•˜๋Š” ์‹œ๊ฐ„์ด ๋„ˆ๋ฌด ์˜ค๋ž˜๊ฑธ๋ ค์„œ ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๋ฐฉ๋ฒ•์œผ๋กœ ์‹ค์‹œ๊ฐ„ ์ถ”๋ก ์„ ๊ฐ€๋Šฅํ•˜๊ฒŒ ํ•˜์˜€๋‹ค.

    1. YOLO ์ž…๋ ฅ ์ด๋ฏธ์ง€ ํฌ๊ธฐ ๊ฐ์†Œ (640, 480) -> (256,192)
    2. GPU ๋Œ€๋ฆฌ์ž ์‚ฌ์šฉ
    3. ๋ฉ€ํ‹ฐ์Šค๋ ˆ๋”ฉ
  • ์ตœ์ข… ๋ชจ๋ธ๋ณ„ & ์ „์ฒด ์ถ”๋ก ์‹œ๊ฐ„

    ๋ชจ๋ธ ์ถ”๋ก ์‹œ๊ฐ„(millisecond)
    ๋ฒˆํ˜ธํŒ ํƒ์ง€ ๋ชจ๋ธ 45
    ๊ผญ์ง“์  ์˜ˆ์ธก ๋ชจ๋ธ 82
    ๊ธ€์ž ๋ชจ๋ธ 86
  • ์–ดํ”Œ๋ฆฌ์ผ€์ด์…˜ ์˜ˆ

    ์˜ˆ์‹œ1 ์˜ˆ์‹œ2
    ์˜ˆ์‹œ1 ์˜ˆ์‹œ2

7. Google Play์— ์–ดํ”Œ๋ฆฌ์ผ€์ด์…˜ ๋“ฑ๋ก

๋‹ค์šด๋กœ๋“œ:

์„ค์น˜ ์ „ ์„ค์น˜ ํ›„
์˜ˆ์‹œ ์˜ˆ์‹œ2
Build an Amazon SageMaker Pipeline to Transform Raw Texts to A Knowledge Graph

Build an Amazon SageMaker Pipeline to Transform Raw Texts to A Knowledge Graph This repository provides a pipeline to create a knowledge graph from ra

AWS Samples 3 Jan 01, 2022
General neural ODE and DAE modules for power system dynamic modeling.

Py_PSNODE General neural ODE and DAE modules for power system dynamic modeling. The PyTorch-based ODE solver is developed based on torchdiffeq. Sample

14 Dec 31, 2022
Speckle-free Holography with Partially Coherent Light Sources and Camera-in-the-loop Calibration

Speckle-free Holography with Partially Coherent Light Sources and Camera-in-the-loop Calibration Project Page | Paper Yifan Peng*, Suyeon Choi*, Jongh

Stanford Computational Imaging Lab 19 Dec 11, 2022
[NeurIPS 2020] Code for the paper "Balanced Meta-Softmax for Long-Tailed Visual Recognition"

Balanced Meta-Softmax Code for the paper Balanced Meta-Softmax for Long-Tailed Visual Recognition Jiawei Ren, Cunjun Yu, Shunan Sheng, Xiao Ma, Haiyu

Jiawei Ren 65 Dec 21, 2022
TumorInsight is a Brain Tumor Detection and Classification model built using RESNET50 architecture.

A Brain Tumor Detection and Classification Model built using RESNET50 architecture. The model is also deployed as a web application using Flask framework.

Pranav Khurana 0 Aug 17, 2021
Cupytorch - A small framework mimics PyTorch using CuPy or NumPy

CuPyTorch CuPyTorchๆ˜ฏไธ€ไธชๅฐๅž‹PyTorch๏ผŒๅๅญ—ๆฅๆบไบŽ๏ผš ไธๅŒไบŽๅทฒๆœ‰็š„ๅ‡ ไธชไฝฟ็”จNumPyๅฎž็ŽฐPyTorch็š„ๅผ€ๆบ้กน็›ฎ๏ผŒๆœฌ้กน็›ฎ้€š่ฟ‡CuPyๆ”ฏๆŒ

Xingkai Yu 23 Aug 17, 2022
Server files for UltimateLabeling

UltimateLabeling server files Server files for UltimateLabeling. git clone https://github.com/alexandre01/UltimateLabeling_server.git cd UltimateLabel

Alexandre Carlier 4 Oct 10, 2022
Neural Tangent Generalization Attacks (NTGA)

Neural Tangent Generalization Attacks (NTGA) ICML 2021 Video | Paper | Quickstart | Results | Unlearnable Datasets | Competitions | Citation Overview

Chia-Hung Yuan 34 Nov 25, 2022
BridgeGAN - Tensorflow implementation of Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation.

Bridging the Gap between Label- and Reference based Synthesis(ICCV 2021) Tensorflow implementation of Bridging the Gap between Label- and Reference-ba

huangqiusheng 8 Jul 13, 2022
Spatially-Adaptive Pixelwise Networks for Fast Image Translation, CVPR 2021

Image Translation with ASAPNets Spatially-Adaptive Pixelwise Networks for Fast Image Translation, CVPR 2021 Webpage | Paper | Video Installation insta

Tamar Rott Shaham 100 Dec 28, 2022
KITTI-360 Annotation Tool is a framework that developed based on python(cherrypy + jinja2 + sqlite3) as the server end and javascript + WebGL as the front end.

KITTI-360 Annotation Tool is a framework that developed based on python(cherrypy + jinja2 + sqlite3) as the server end and javascript + WebGL as the front end.

86 Dec 12, 2022
IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID,

Intermediate Domain Module (IDM) This repository is the official implementation for IDM: An Intermediate Domain Module for Domain Adaptive Person Re-I

Yongxing Dai 87 Nov 22, 2022
Adversarial vulnerability of powerful near out-of-distribution detection

Adversarial vulnerability of powerful near out-of-distribution detection by Stanislav Fort In this repository we're collecting replications for the ke

Stanislav Fort 9 Aug 30, 2022
Project of 'TBEFN: A Two-branch Exposure-fusion Network for Low-light Image Enhancement '

TBEFN: A Two-branch Exposure-fusion Network for Low-light Image Enhancement Codes for TMM20 paper "TBEFN: A Two-branch Exposure-fusion Network for Low

KUN LU 31 Nov 06, 2022
2021 CCF BDCI ๅ…จๅ›ฝไฟกๆฏๆฃ€็ดขๆŒ‘ๆˆ˜ๆฏ๏ผˆCCIR-Cup๏ผ‰ๆ™บ่ƒฝไบบๆœบไบคไบ’่‡ช็„ถ่ฏญ่จ€็†่งฃ่ต›้“็ฌฌไบŒๅๅ‚่ต›่งฃๅ†ณๆ–นๆกˆ

2021 CCF BDCI ๅ…จๅ›ฝไฟกๆฏๆฃ€็ดขๆŒ‘ๆˆ˜ๆฏ(CCIR-Cup) ๆ™บ่ƒฝไบบๆœบไบคไบ’่‡ช็„ถ่ฏญ่จ€็†่งฃ่ต›้“็ฌฌไบŒๅ่งฃๅ†ณๆ–นๆกˆ ๆฏ”่ต›็ฝ‘ๅ€: CCIR-Cup-ๆ™บ่ƒฝไบบๆœบไบคไบ’่‡ช็„ถ่ฏญ่จ€็†่งฃ 1.ไพ่ต–็Žฏๅขƒ๏ผš python==3.8 torch==1.7.1+cu110 numpy==1.19.2 transformers=

JinXiang 22 Oct 29, 2022
Organseg dags - The repository contains the codebase for multi-organ segmentation with directed acyclic graphs (DAGs) in CT.

Organseg dags - The repository contains the codebase for multi-organ segmentation with directed acyclic graphs (DAGs) in CT.

yzf 1 Jun 12, 2022
LBBA-boosted WSOD

LBBA-boosted WSOD Summary Our code is based on ruotianluo/pytorch-faster-rcnn and WSCDN Sincerely thanks for your resources. Newer version of our code

Martin Dong 20 Sep 19, 2022
PyTorch implementation of neural style transfer algorithm

neural-style-pt This is a PyTorch implementation of the paper A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias

770 Jan 02, 2023
In this project I played with mlflow, streamlit and fastapi to create a training and prediction app on digits

Fastapi + MLflow + streamlit Setup env. I hope I covered all. pip install -r requirements.txt Start app Go in the root dir and run these Streamlit str

76 Nov 23, 2022
For IBM Quantum Challenge Africa 2021, 9 September (07:00 UTC) - 20 September (23:00 UTC).

IBM Quantum Challenge Africa 2021 To ensure Africa is able to apply quantum computing to solve problems relevant to the continent, the IBM Research La

Qiskit Community 48 Dec 25, 2022