Scala basic tutorial -- 12 -- Reading and writing data

Scala Basic course –12– Read and write data

Chapter goal

1. master Source The function of reading data
2. Master the function of writing data
3. Master student transcript cases
4. Reading data

---

1. Reading data

stay Scala Linguistic Source Singleton object in in , Provides some very convenient methods , So that developers can quickly from the specified data source ( text file , URL Address, etc ) Get data in , In the use of Source Singleton object Before , Pilot package required , namely import scala.io.Source .

1.1 According to the line read

We can use That's ok In units of , To read the data in the data source , The return value is one Iterator type The object of . And then through toArray, toList Method , Put these data into an array or list .

Be careful : Source Class extends from Iterator[Char]

Format

//1.  Get data source file object .
val source:BufferedSource = Source.fromFile(" Path of data source file "," Encoding table ")
//2.  Read data in behavioral units .
val lines:Iterator[String] = source.getLines()
//3.  Encapsulate the read data into a list .
val list1:List[String] = lines.toList
//4.  Don't forget to close Source object .
source.close

demand

1. Create... Under the current project data Folder , And create 1.txt text file , The contents of the document are as follows :

 study hard  ,  Day day up !
Hadoop, Zookeeper, Flume, Spark
Flink, Sqoop, HBase
 Choose big data ,  Achieve your lifelong dream 

2. Read the data in the text file in behavioral units , And print the results .

Reference code

import scala.io.Source
// Case study :  Demonstrate reading lines .
object ClassDemo01 {
    
	def main(args: Array[String]): Unit = {
    
		//1.  Get data source object .
		val source = Source.fromFile("./data/1.txt")
		//2. adopt getLines() Method ,  Get the data in the file line by line .
		var lines: Iterator[String] = source.getLines()
		//3.  Encapsulate every piece of data obtained into a list .
		val list1 = lines.toList
		//4.  Print the results 
		for(s <- list1) println(s)
		//5.  Remember to turn off source object .
		source.close()
	}
}

1.2 Read by character

Scala It also provides Read data in characters This way, , This usage is similar to iterators , After reading the data , We can go through hasNext(),next() Method , Flexible access to data .

Format

//1.  Get data source file object .
val source:BufferedSource = Source.fromFile(" Path of data source file "," Encoding table ")
//2.  Read data in characters .
val iter:BufferedIterator[Char] = source.buffered
//3.  Encapsulate the read data into a list .
while(iter.hasNext) {
    
	print(iter.next())  
}
//4.  Don't forget to close Source object .
source.close()

Be careful :
If the file is not very large , We can read it directly into a string .

val str:String = source.mkString

demand

1. Create... Under the current project data Folder , And create 1.txt text file , The contents of the document are as follows :

 study hard  ,  Day day up !
Hadoop, Zookeeper, Flume, Spark
Flink, Sqoop, HBase
 Choose big data ,  Achieve your lifelong dream .

2. Read the data in the text file in behavioral units , And print the results .

Reference code

import scala.io.Source
// Case study :  Demonstrate reading a single character .
object ClassDemo02 {
    
	def main(args: Array[String]): Unit = {
    
		//1.  Get data source object .
		val source = Source.fromFile("./data/1.txt")
		//2.  Get every character in the data source file .
		val iter = source.buffered    // here , source Object usage is equivalent to iterators .
		//3.  adopt hasNext(), next() Method to get data .
		while(iter.hasNext) {
    
			print(iter.next())       // details ,  Don't use it here println(),  Otherwise, the output may not be what you want .
		}
		//4.  adopt mkString Method ,  Directly encapsulate all the data in the file into a string .
		val str = source.mkString
		//5.  Print the results .
		println(str)
		//6.  close source object ,  Saving resource ,  Increase of efficiency .
		source.close()
	}
}

1.3 Read lexical units and numbers

The so-called lexical unit refers to Strings separated by specific symbols , If the data in the data source file is String in numeric form , We can easily get these data directly from the file , for example :

10 2 5
11 2
5 1 3 2

Format

//1.  Get data source file object .
val source:BufferedSource = Source.fromFile(" Path of data source file "," Encoding table ")
//2.  Read lexical units .
// \s For white space characters ( Space , \t, \r, \n etc. )
val arr:Array[String] = source.mkString.split("\\s+") 
//3.  Convert the string to the corresponding integer 
val num = strNumber.map(_.toInt)
//4.  Don't forget to close Source object .
source.clo

demand

1. Create... Under the current project data Folder , And create 2.txt text file , In the file

10 2 5
11 2
5 1 3 2

2. Read all integers in the file , Add it to 1 after , Print the results to the console .

Reference code

import scala.io.Source
// Case study :  Read lexical units and numbers .
object ClassDemo03 {
    
	def main(args: Array[String]): Unit = {
    
		 val source = Source.fromFile("./data/2.txt")
		 // \s For white space characters ( Space , \t, \r, \n etc. )
		 val strNumber = source.mkString.split("\\s+") 
		 // Convert the string to the corresponding integer 
		 val num = strNumber.map(_.toInt)
		 for(a <- num) println(a + 1)
	}
}

1.4 from URL Or read data from other sources

Scala Provides a way to , Let's go directly from the designated URL route , Or other sources ( for example : Specific string ) Read data directly from .

Format

• from URL Read data in address

//1.  Get data source file object .
val source = Source.fromURL("http://www.itcast.cn")
//2.  Encapsulate the data into a string and print .
println(source.mkString

• Read data from other sources

//1.  Get data source file object .
val str = Source.fromString(" Black horse programmer ")
println(str.getLines())

demand

1. Read Baidu (https://www.baidu.com/) Page data , And print the results .
2. Read the string directly Black horse programmer , And print the results .

Reference code

import scala.io.Source

// Case study :  from URL Or read data from other sources 
object ClassDemo04 {
    
  def main(args: Array[String]): Unit = {
    
    val source = Source.fromURL("https://www.baidu.com/")
    println(source.mkString)
    val str = Source.fromString(" Baidu once you know ")
    println(str.getLines())
  }
}

1.5 Read binary

Scala There is no way to read binary files , We need to pass Java Class library to implement .

demand

Of known projects data There is... Under the folder 05.png This picture , Please read the image data , And print the number of bytes read to the console .

Reference code

//  Case study :  Read binary data .
object ClassDemo05 {
    
  def main(args: Array[String]): Unit = {
    
    val file = new File("./data/04.png")
    val fis = new FileInputStream(file)
    val bys = new Array[Byte](file.length().toInt)
    fis.read(bys)
    fis.close()
    println(bys.length)
  }
}

2. Write data

Scala There is no built-in support for writing files , To write data to a file , Still need to use Java Class library of .

2.1 Write the specified data to the file

demand

Under the project data The folder 3.txt In the text file , Write a sentence , The contents are as follows :

 study hard  ,
 Day day up !

Reference code

//  Case study :  Write data to a text file .
object ClassDemo06 {
    
  def main(args: Array[String]): Unit = {
    
    val pw = new FileOutputStream("./data/3.txt")
    pw.write(" study hard ,\r\n".getBytes())
    pw.write(" Day day up !".getBytes())
    pw.close()
  }
}

2.2 Serialization and deserialization

stay Scala in , If you want to transfer objects to other virtual machines , Or temporary storage , You can go through Serialization and deserialization To implement the .

• serialize : The process of writing objects to a file .
• Deserialization : The process of loading objects from files .

Be careful : If the object of a class wants to realize serialization and deserialization , Then this class must inherit Serializable Trait .

demand :

1. Define the sample class Person, The attributes are name and age .
2. establish Person The object of the sample class p.
3. The object is serialized p Write to... Under item data Under folder 4.txt In the text file .
4. Through deserialization operation, from the... Under the project data Under folder 4.txt In file , Read the object p

Reference code

//  Case study :  Demonstrates serialization and deserialization operations .
object ClassDemo07 {
    
  case class Person(var name:String, var age:Int)
  def main(args: Array[String]): Unit = {
    
    // Serialization operation .
    /*val p = new Person(" Zhang San ", 23) val oos = new ObjectOutputStream(new FileOutputStream("./data/4.txt")) oos.writeObject(p) oos.close()*/
    // Deserialization operation .
    val ois = new ObjectInputStream(new FileInputStream("./data/4.txt"))
    var p: Person = ois.readObject().asInstanceOf[Person]
    println(p)
  }
}

3. Case study : Student transcript

3.1 summary

1 . ... under known projects data The folder student.txt In the text file , Record the achievements of some students , as follows :

The format is : full name Chinese achievement Math scores English scores

 Zhang San  37 90 100
 Li Si  90 73 81
 Wang Wu  60 90 76
 Zhao Liu  89 21 72
 Panax notoginseng  100 1

2. Rank the students in descending order according to their total scores , according to full name Chinese achievement Math scores English scores Total score The format of , Write the data to data The folder stu.txt In file .

3.2 Purpose

Investigate flow , Sample class , And functional programming Related content .

3.3 step

1. Define the sample class Person, The attribute is : full name , Chinese achievement , Math scores , English scores , And there is a way to get the total score in this category .
2. Read the specified file (./data/student.txt) All the data in , And encapsulate it into List In the list .
3. Define variable lists ListBuffer[Student], It is used to record the information of all students .
4. Traverse the data obtained in the second step , Encapsulate it as Person After the object of the class , To add to ListBuffer in .
5. Right. 4 Sort the data obtained in step , And turn it into List list .
6. In the specified format , adopt BufferWriter Write the sorted data to the destination file (./data/stu.txt)
7. Close flow object .

3.4 Reference code

//  Case study :  Arrange in descending order according to the total score of the trainees .
object ClassDemo08 {
    
  //1.  Define the sample class Person,  attribute :  full name ,  Chinese achievement ,  Math scores ,  English scores ,  And there is a way to get the total score in this category .
  case class Student(name:String, chinese:Int, math:Int, english:Int) {
    
    def getSum() = chinese + math + english
  }
  def main(args: Array[String]): Unit = {
    
    //2.  Get data source file object .
    val source = Source.fromFile("./data/student.txt")
    //3.  Read the specified file (./data/student.txt) All the data in ,  And encapsulate it into List In the list .
    var studentList: Iterator[List[String]] = source.getLines().map(_.split("
    ")).map(_.toList)
    //4.  Define variable lists ListBuffer[Student],  It is used to record the information of all students .
    val list = new ListBuffer[Student]()
    //5.  Traverse the data obtained in the second step ,  Encapsulate it as Person After the object of the class ,  To add to ListBuffer in .
    for(s <- studentList) {
    
      list += Student(s(0), s(1).toInt, s(2).toInt, s(3).toInt)
    }
    //6.  Right. 5 Sort the data obtained in step ,  And turn it into List list .
    val sortList = list.sortBy(_.getSum()).reverse.toList
    //7.  In the specified format ,  adopt BufferWriter Write the sorted data to the destination file (./data/stu.txt)
    val bw = new BufferedWriter(new FileWriter("./data/stu.txt"))
    for(s <- sortList) bw.write(s"${
    s.name} ${
    s.chinese} ${
    s.math} ${
    s.english}
    ${
    s.getSum()}\r\n")
    //8.  Close flow object 
    bw.close()
  }
}